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PRESENTING A NEW JOURi^j/^L 




The interest and activity in the field of 
human characteristics have never been greater than 
with this thought that the editors present the first 
cational and Psychological Measurement. E^y^g 
tions, government, and industry are all giving institu- 

tion to methods of evaluation aimed at determinino- tl 
promise of the individual. Improved methods in 
are being developed and significant research Ic , 
many fields. 


In spite of this rising interest, measurement is ■ 
child. The contributions of measurement theory and 
found expression in the publications devoted prima^T^'"''^'^ 
fields. Nowhere has there been a common meetinp- ^ 
the exchange of ideas from area to area except f 
technically inclined. 


more 


Yet there are measurement problems of practical 
diate concern which are common to many fieigg imme- 

of estimating future success is common, for example t ^ 
of helping young people choose appropriate vocations*^ 
ing employees, of admitting students to educational 
and assigning draftees to jobs in the Army. institutions 

The limited interchange of ideas and techniques ' 

ment probably can be explained by the fact that there 

no single journal which could be counted upon to te ^ 

developments and to serve as a forum for the disi'no ■ ” 

i ^‘scussion of prob- 



lems. It is our purpose to remedy this situation. The pages of 
Educational and Psychological Measurement will be open to con- 
tributions from all fields in which techniques of human measure- 
ment are used. Each issue of the journal will have departments 
devoted to news and abstracts of recent literatuie. Future i.s.sues 
will also carry a section on new tests. 

It is hoped that the articles in the journal will not only be 
of interest to readers in the specific areas from wliich the article.s 
come, but that they will be suggestive of improved procedure.s 
elsewhere. 


Washington, D. C. 
December 23, 1940. 


G. F. K. 



THE EVALUATION OF VOCATIONAL AND 
EDUCATIONAL COUNSELING: 

A CRITIQUE OF THE METHODOLOGY OF 
EXPERIMENTS^ 

E. G. WILLIAMSON AND E. S. BORDIN 
Univeisity of Minnesota 

With increasing attempts to systematize the concepts of coun- 
seling, to describe its techniques* and to delineate its objectives, the 
need for evaluative studies has become more insistent. Descriptions 
of programs of vocational and educational counseling usually close 
with a summary statement that fuithcr improvement in this field 
is dependent upon evaluative studies (40: chap. XXVII, 42, 43: 
chap. IX, 441). In other words, cuirently used techniques of coun- 
seling must be subjected to scrutiny and evaluation in order that 
more effective ones may be developed. Thus a fertile field for ex- 
perimentation may be found in this phase of student pen'Onncl work. 

Restricting Conditions 

A review of the peculiar conditions of this field of applied psy- 
chology is in order and should precede attempts to experiment This 
paper will attempt to summarize, in a critical and systematic man- 
ner, the assumptions, criteria, methods of measuring outcomes, and 
possible experimental designs involved in the evaluation of educa- 
tional and vocational counseling. The tieatment of personality, 
social, family and other types of students' problems will be con- 
sidered only in relationship to educational and vocational adjust- 
ment. The evaluation of these other types of counseling— usually 
called personality counseling — should be the subject of another 
paper. 

When we speak of counseling, we refer to individualized efforts 
to help students discover vocational assets and disabilities and to 
plan an appropriate training program. The making of such an in- 
ventory of potentialities must be preceded by the collection and use 
of evidence of abilities, interests and motivations* The techniques 
involved in collecting, refining and using evidence have been de- 
sciibed elsewhere (40: chap. III). 

^The leport of a statistical evaluation of clinical counseling by the 
same authois will appear in the next issue of this journal. 

fSee pages 22-24 for references. 
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For purposes of evaluation experiments, it is necessary also to 
agree as to what counseling is not, to define it negatively. For 
example, we cannot accept the assumption that testing alone or that 
statistical prediction is counseling. Such would seem to be the 
assumption of Thorndike’s 1934 study (35) as an evaluation of 
counseling techniques. On the other hand, attempts to define coun- 
seling as self-analysis (by students) or as diagnosis based alone 
upon impressions, student hopes and interview data (by counselors) 
are equally unacceptable. Counseling must be based upon an under- 
standing of the student; but the counselor does more than make a 
diagnosis or prediction. Counseling is the process of helping the 
student to plan, and to utilize his assets. 

Progress toward adequate evaluation of counseling has been 
impeded by two types of attitudes held by some personnel workers. 
Some counselors evaluate by means of arm-chair methods. That is, 
the effectiveness and general worth of counseling is held to be self- 
evident. These persons reason that the general methodology of 
guidance must be effective because it appears to be an appropriate 
method of dealing with serious and widespread maladjustment 
among youth. Other personnel workers appear to believe that coun- 
seling cannot be evaluated. They maintain that the counseling 
process is so personal and individual that any attempt by the coun- 
selor to study it will impair his efficiency as a counselor and will 
create an artificial situation which will not even remotely resemble 
the real counseling relationship. 


On the other hand, those who believe that counseling can and 
should be evaluated have taken one of three approaches. First, 
there is the approach which clings to traditional statistical meth- 
odology m utilizing only those criteria that are objectively quanti- 
fiable. This approach is based upon the premise that a straight- 
forward statistical analysis of such data as grades, years in college 
number of jobs held or wages earned, are sufficient criteria for 

non-statistical case study methods of evaluation. The third ao- 
proach attempts to avoid the objections to the other two methods 
by using various objective and systematically derived criteria 
w ich are combined by means of impartial judgmental treatment 
m contrast with statistical summations. 


Th, assumptions underlying criteria should be mode eKolicit 
Implioit assumpdous have been the source of error in ‘ i 
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guidance. Here again the interpretationb of ThorncUke’h .study (35) 
serve as an example, although others might also be used. The con- 
clusion, drawn by many from Thorndike's study, that coun.scling 
was low in effectiveness, would not be objectionable if the interpre- 
ters had indicated that by guidance they mean .stati.sticnl prediction 
of fragmented criteria. In speaking of prediction of fragmented cri- 
teria we refer to the fact that many reseaich workers lose sight of 
the possibility that one datum often has difTcrent meaning and sig- 
nificance for different students. If such is the case, and we have 
every reason to believe that it is, then any attempt to u.se these bits 
of information either separately or in a rigid arithmetic combina- 
tion may obscure the actual outcomes of counseling. 

The supposition that specific objectives, such as an increase in 
academic achievement, will neccssr.iily be common to all the ca.ses 
in an experimental population musi a^so bo cxaimiu'd. If we cannot 
accept the supposition, then we must consider the possibility that 
the use of what is at best a partially applicable ciiterion is likely to 
reveal only slight differences, if any at all. For example, a low 
aptitude student who had been successfully counseled into with- 
drawing from college cannot be included in an experiment de.signed 
to reveal the effectiveness of counseling in increasing grades. 

There are two other considerations of this type that the careful 
research worker must consider in planning an ellective evaluative 
experiment. First he must realise that in otdet to evaluate a pro- 
gram of action, it must be carried out, I he student must do some- 
thing following counseling nr oulei to make evaluation po.ssible. 
A physician might just as well attempt to discover the effective- 
ness of his medicine when his patient has taken it home and placed 
it unused in his medicine cabinet. Secondly, a counselor may 
change a student s attitudes, but these must be revealed in observ- 
able or measurable behavior or they cannot be evaluated. Any out- 
come that IS beyond the scope of some means of dependable olxser- 
vation is one that cannot be dealt with and therefore must he re- 
jected by those who require more than blind faith. 

The question of the optimum time interval for evaluation ia one 
that needs further investigation before much [vrogres.s can be made 
m evaluation experinieius. It is possilile that the optimum time 
interval will vary for each individual in any experimental group; 
or perhaps the longer the intervening time, the greater the possi- 
bility for the intrusion of other influences that may tend to mini- 
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mize the effects of counseling'. Some influences may facilitate ad- 
justment subsequent to counseling; others may cause malacljusN 
ment. Even though counseJing results in a distinct separation of 
counseled from non-counseled in terms of subsequent adjustment, 
the randomization of subsequent influences may cause a regression 
toward the mean for both groups. 

The scant knowledge of specific counseling techniques has 
forced us to study the effectiveness of the total process. If certain 
techniques neutralize the effect of others, then the gross results 
would be negligible. As specific techniques are isolated and de- 
scribed, then new types of evaluation studies may replace the pres- 
ent gross experiments. Such studies, however, would not appear to 
be possible until more adequate descriptions of techniques are made 
available by those who actually counsel students. 


Formulating Hypotheses 

Counseling can be evaluated only if certain outcomes or criteria 
of effectiveness are assumed to result from the counseling process. 
These assumptions must be formulated as hypotheses to be “tested" 
by experimental and statistical analyses. But a second considera- 
tion IS of equal importance. We must determine not only the resuJtK 
of counseling but, as in all scientific studies, the conditions under 
which these outcomes will be produced. We must answer this sec- 
ond question in terms of what kinds of counseling, what techniques, 
what types of counselors and work with what types of studciitb will 
produce certain outcomes. Our problem, broadly speaking, then 
becomes, "'What counseling techniques (and conditions) wiii pro- 
duce what types of results with what types of students^^ 

^ Most counselors have empirically derived opinions, luinches and 
judgments as to what outcomes or effects they and the students 

outcomes are intangible 
and difficult to formulate as well as difficult to set up in an experi- 

man al design, Wa may, howaver, achieve some degree of agree- 
mmptiSa-''”’’"*" ‘»P«™entation, on the following a.s- 

Effective counseling will lead to or result in- 

range) aptitude than he possesses (actually and potentially). 
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3. The student will make reasonable progress toward this 
goal (in training school), 

4. The student will be “satisfied” (further motivated) by 
that progress and with his chosen goal. 

In order to achieve these outcomes it is necessary that: 

1. The counselor secures the student’s cooperation (rap- 
port in the broad sense) in choosing (orienting himself 
toward) a goal and the means to it; the desire to assay his 
assets and interests. 

2. The student generates enthusiasm to use his assets in 
attempting to secure relevant training and to achieve the 
chosen goal. 

3. The student uses his aptitudes skillfully in securing 
training in school. 

4. The counselor and the student are able to alleviate, 
relieve or remedy pressures and disabilities — 'family, financial, 
emotional, etc. — which interfere with or prevent the eager 
and skillful use of aptitudes and the choice of an appiopriate 
goal. 

5. If these pressures or disabilities are too serious for 
the counselor to cope with, then use is made of specialized 
personnel workers, 

6. The appropriate or reasonably approKimate type of 
training is available to the student. 

The above possible outcomes may be the direct or indirect* 
immediate or long-term outcomes of counseling. They may reveal 
themselves or be observed indirectly and not always by means of 
the student’s verbal report to the counselor. For example, the stu- 
dent’s orientation may be revealed in his classroom grades. Some 
outcomes may be general in nature (results of any type of counsel- 
ing technique), and others may be highly specific. Likewise some 
techniques may produce one or more of the above outcomes when 
used with any type of student having any type of problem, Other 
techniques may be highly specific. Much experimentation needs to 
be done before we can answer these subsidiary questions. It is most 
likely that counseling cannot be equally effective with all types of 
students and all types of conditions. 

Experimental Designs 

Drawing upon empirical knowledge, we may describe the gen- 
eral outlines of a number of possible experiments which should 
reveal some of the outcomes of counseling. Wc shall restrict our- 
selves to the following possible criteria ; academic achievement, 
appropriate choices, cooperation, satisfaction, success, quality of 
case work, predictive efficiency, composite criteria. 

Academic Achievement. The emphasis placed upon grades in 
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educational circles has necessarily established them as the most 
used criterion of the effectiveness of counseling. Most collegeb and 
universities drop unsatisfactory students on the basis of their aver- 
age grades, and reward those who achieve high marks, There are 
two methods of experimental design applicable: the comparison of 
the student’s grade average before and after counseling (2, 4, 8, 19 
24, 43: chap. IX); or a comparison of the average grade of coun- 
seled students with that of non-counscled students who have been 
matched for such characteristics as age, sex, level of ability, size 
and type of high school and high-school grades (13, I. '5, 20, 27 
41, 42). 

Both methods of control have definite weaknesses, First of all 
it must be emphasized that grades are patently only one of the pos- 
sible desirable outcomes of counseling. In addition their icliability 
and validity, as a measure of scholastic achievement, have been 
seriously questioned. Of more importance are the dissimilarities in 
patterns of subjects taken by different students. This condition 
makes the criterion of average grade a shifting scale wlio.se com- 
parability from student to student is questionable. Moreover, in 
cases where the student has been successfully advised to leave col- 
lege there will be no subsequent grades to evaluate, In the case of 
students counseled before matriculating in college, where no pro- 
counseling grades are available, this method is not at all applicable. 

The method of control by matching is a traditional one in scien- 
tific experimentation. It theoretically provides us with a compar- 
able population for comparing the effect of counseling with the 
effect of normal” (or random) conditions. At the present lime 
however, it is impossible to match individuals on the very factors 
that may be of importance, e.g„ motivation, personality o, emo'- 
tional stability. In addition, it is difficult to collect a L.onM, 
number of cases which will be matchable. While the method of 
internal control, i.e.. comparing grades before and after couns^v 
does away with the matching problem, it leaves indeterminate tffe 

Bv KWevement lasts is a possible altoraa- 
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their objectives and college subjects this factor of heterogencit] 
will be a possible disturbance in the use of scholastic criteria 0 
counseling effectiveness. Experiments should be made to determinf 
the possible relevancy and validity of this type of ciiterion. 

Educational and Vocational Choices. When evaluating it 
terms of educational and vocational choices it may he assunict 
that the individual will achieve a more satisfactory hfe adju.st^ 
ment if he sets goals for himself that are neither too high not 
too low for his potentialities (18, 32). Thus the task of tli« 
counselor is conceived to be, in part at least, to hring^ about con- 
gruence between those two factors. 

For any case we may compare the student's statement of Ins 
objectives with his potentialities as judged from test data and rele- 
vant tryout expel iences. The judgment of the degree to which bet- 
ter alignment has been achieved as a result of counseling may be 
made by the counselor himself, by an outsider who leads the case 
notes, by the student, or by all three persons. In favor of the former 
procedure, one may contend that there are often subliminal data 
not included m the case record which would make the cotinsclor's 
judgment most accurate. On the other hand, we may encounter 
difficulty in separating judgment from de.sire since the counsdor is 
not disinterested in the outcome. An added difficulty with this type 
of criterion is the frequency of student cases in which a temporarily 
uncertain choice is the most desirable outcome of counseling. 

An induect measure of thi.s criterion may lie used if we assume 
that more information on educational and vocational topics will 
lead to a greater probability of congruence heuveen aspiratioiis and 
potentialities. It seems legitimate to e,xpect the clinical counaclor 
to aid the student in acquiring such information, altliough this type 
of function has usually been involved in group guidance procedures. 
For the appraisal of these two types of outcomes, tests and inven- 
tories of the Kefauver-Hand type may be used (17). By these 
means it may be possible to determine whether counseled students 
have more information on which to base their educational and voca- 
tional decisions than they had before couiuseling or than is pos- 
sessed by a matched uncoimselcd control group. Since the mere 
possession of occupational and educational information is not a 
major objective of counseling, experiment.^ are needed to deter- 
mine the relationship between the possession of such information 
and the appropriateness of the choices made by students Such 
crucial experiments have not yet been made in support of the 
relevancy for counseling of courses in occupational information. 


11 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


Cooperation with the Counselor, This criterion is based upon 
the premise that effective results of counseling cannot be achieved 
unless the counselor is en rapport with the student, The fact that 
the student receives the advice of the counselor cooperatively is 
taken as an indication of rapport and therefore as a criterion of the 
effectiveness of counseling. Viteles seems to go even fuithcr when 
he says; “That advice is followed is probably in itself an evidence 
of satisfactory adjustment" (38: p. 75). Such a contention needs to 
be evaluated experimentally. We would reject any attempt on a 
physician’s part to prove the efficacy of a particular medical treat** 
ment by means of evidence that the patient cooperates in submit- 
ting to that treatment. We would certainly withhold judgment 
until we ascertained whether his patient eventually had recovered 
or died. Cooperation is a desired outcome of counseling but 
chieSy as a means or condition necessary to other more basic out- 
comes, In this sense it is a preparatory outcome or criterion of 
counseling effectiveness. 

The measurement of this criterion would be expressed in terms 
of the percentage of the group counseled that had shown various 
degrees of cooperation. Such a result is difficult to interpret since 
there is no standard for determining what either a statistically or a 
socially significant percentage would be. Further experimentation 
and experience would, of course, provide data for deiiving such a 
standard. 


The Student's Satisfaction. Satisfaction of the student is 
deemed to be a desirable outcome of counseling. This .satisfaction 
may embrace his educational and vocational objectives, the counsel- 
ing assistance, and finally the job that he ultimately secures. The 
student’s satisfaction with any of the three may be inferred from 
his verbal report, either on an interview basis or by means of an 
attitude test Obviously many subtle or delayed satisfactions may 
not be readily observed or felt by the student Dissatisfaction which 
results from frustration may be, and oftentimes is, followed by later 
reconciliation to substitute adjustments. 

Concerning satisfaction with educational and vocational obiec- 
methods of control may be used, The salis- 

or thlV ■ ? . measured before and after counseling 

or the saUsfactmn of a counseled group may be compared to that of 

assisCrS satisfaction with counseling 

assistance (25, 39) neither of these methods is possible. To mean- 

ure a student’s satisfaction with counseling assistance before he has 
been counseled or when he has not been counseled wouldTe t an- 
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ingless. We can only determine the percentage of students who ex- 
pressed degrees of satisfaction with the counseling assistance 
received and compare the results for two or inoie counseled gtoups. 
In a sense this criterion is usable to determine which of two or 
more counseling methods, or counselors, is more effective. 

The systematic and quantitative data provided by the attitude 
scale technique have not as yet been exploited in the evaluation of 
counseling. There are three types of attitude scales that may he 
used. First, a scale measuring the stude^t^s attitude toward the 
school and his educational training. Bell has already described 
such a scale for high school students (3 i p. 117-23). Second, a scale 
measuring the student’s attitude toward his vocational objectives. 
Remmers has constructed such a scale and has used it in the apprai- 
sal of the effectiveness of group guidance (28, 29). Third, a scale 
measuring the student’s attitude toward the counselor and the 
counseling assistance. This type of scale has had practically no 
application. In fact we have found only two instances of its use 
reported in the literature (14, 23). The usual appioach has been 
through the report of the individual to direct questioning. 

While the studeiit’s report is the easiest way to determine satis- 
faction and cannot be ignored as one type of satisfaction response, 
it has many weaknesses. For example, it may conceal real dissatis- 
faction behind a rationalization process. It may he a reOection of 
dissatisfaction in some other area than education or vocation, c.g., 
social, recreational, sex. The desire to please the couiuielor because 
of fixation or gratefulness may lead to a report of satisfaction. In 
some cases it seems too much to expect a feeling of complete satis- 
faction even with the most successful counseling, since a counselor 
cannot be expected to overcome the false hopes of a lifetime in a 
relatively short period of time, If the individual’s stratum of society 
requires a level of aspiration far beyond his capabilities, the coun- 
selor cannot be expected to bring about complete and immediate 
satisfaction. 

Satisfaction with a job has been the most frequently used cri- 
terion of the effectiveness of vocational counseling (4, 5, G, 16, 22, 
23, 25, 30), In addition to the direct report of the student, scores 
on the Hoppock Job Satisfaction Blank and the number of voluntary 
shifts in jobs have been used as measures of job satisfaction. All of 
these criteria lend themselves to the use of both an internal and a 
matched control. But many objections are encountered to satisfac- 
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tion with the job, measured in any manner. Job dissatisfaction may 
reflect dissatisfaction with the low starting salaries which are char- 
acteristic of most jobs rather than with the occupational choice 
resulting from counseling. The dissatisfaction may also be caused 
by local conditions on the job, e.g., an unpleasant supervisor, un- 
companionable workmates, instead of maladjustment to the work 
involved. Likewise there are special objections to the use of shifts 
in employment as a criterion, since it is difficult to distinguish a 
voluntary from a forced shift. A shift, as measured, is an alLor- 
none process and time does not allow for a measure of degrees of 
satisfaction or promotion. 

The method of internal control with job satisfaction as the cri- 
terion is different from the one previously outlined. In this case 
before-after comparisons are not applicable. Instead, those who are 
in an advised occupation are compared to those who are not or 
with those who are in an occupation not discussed with the coun- 
selor. For this control to be meaningful the categories of occupa- 
tions must be broadly interpreted according to their general 
functions. 

Success on the Job. This criterion assumes that effective 
counseling should lead students to seek and secure jobs in which 
they can be successful. It can be measured by employer's lepotts, 
number of advancements, number of forced shifts and wages earned. 
The controls applicable are the same as those for the criterion of 
job satisfaction (4, 9, 16, 22, 25, 30). 

The use of a success criterion has at least four general weak- 
nesses. First of all, success is a relative matter, relative to the stu- 
dent's ambitions and to the reactions of his social group to his 
achievements. Secondly, success may come years later with many 
other factors, unrelated to the original counseling, intervening to 
cause it. Success m school is a more immediate adjustment the 
student must make before the vocational adjustment is necessary. 
Thirdly, some students advance vocationally more quickly because 
of aids from parents or friends and not because of counseling* 
Finally, this criterion is complicated by the influence of the quality 
of placement work in the senior year of training and is only re- 
motely a criterion of counseling in the freshman year. 

Each of the methods of estimating this criterion has been seri- 
ously criticized (33, 34). Employer’s reports may be subject to 
error because of the influence of an adverse personal relationship 
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between employer and employee unrelated to quality of work, be- 
cause of the state of the labor market or because of atypical suc- 
cesses or failures at the time of the foUow-up interview or ques- 
tionnaire. Quite often there will be problems of locating the 
employer, especially when the student has experienced a number of 
shifts within a short period of time (9). The absence of standards 
for comparison and the difficulty in securing cooperation arc also 
contributory factors to the unreliability of employers' reports. 

Number of advancements in employment may be unsatisfac- 
tory as a criterion of success because the best occupation for an 
individual may be one in which there are few opportunities for 
advancement. In addition, in most cases advancement occurs over 
a long period of time. The longer the intervening time, the more 
difficult it is to determine whether the original counseling has 
been the decisive factor rather than any of the many intervening 
influences. Likewise, the number of shifts in employment pre- 
sents drawbacks because of the difficulty in distinguishing volun- 
tary from forced shifts and the all-or-none nature of shifts in 
jobs. 

Paterson and Darley (26: p, 19) and, more recently^ Lurie (21) 
have presented evidence which indicates that shifts in jobs may 
not always be reliable indices of the individuaVs adjustment. 
The older study found that the number of job changes did not 
discriminate woikers unemployed early in the depression from 
those unemployed late. Lurie found that workers discharged 
during retrenchment were, as a group, as capable as those 
retained. 

In order for compaiisons on the basis of wages earned to be 
meaningful, it is necessary to compare individuaLs who are work- 
ing on jobs where comparable wage scales prevail, a difficult task. 
Another objection is that wages may reflect extra-individual 
conditions beyond the scope of the counselor's function. 

Quality of Case Work. The type and appropriateness of the 
various procedures and techniques used by the counselor are 
assumed to be the marks of good counseling. Studies using such 
criteria arc, however, to be considered as preparatory to final 
studies of the effectiveness of guidance. It should be recogni^sed, 
however, that unless thorough-going methods are used there is 
little point in making an experimental evaluation (42). 

A critical analysis of the techniques used by the counselor 
and a critical reading of case history and interview notes are the 
most feasible methods to determine their appropriateness (7, 39), 
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An unbiased but well-informed “outside’’ judge would seem to be 
the most desirable agent to perform such an analysis of case 
records. There is a difficulty here in that a well-informed judge 
is likely to be one who has had counseling experience himself 
and is, therefore, unlikely to be free of convictions This method 
is at best a rough measure of whether the counselor used the 
particular techniques judged appropriate by other counselors. No 
measure of the effectiveness of these techniques results from the 
use of this criterion. 

Predictive Efficiency. The efficiency of educational diagnosis 
by the counselor may perhaps be studied more accurately. One 
possible experimental setup would compare the efficiency of pre- 
diction for pre-college cases by the counselor to that of a statis- 
tical predictive equation (5). The problem could be further 
differentiated by comparing predictions made by the counselor on 
the basis of preliminary information, tests, questionnaire infor- 
mation and preliminary interview, with predictions after the first 
counseling interview. This would serve to determine the relative 
importance or validity of the information and impressions col- 
lected in the counseling interview with case data, such as test 
scores, available to the counselor before he confers with the 
student. Another differentiating study would involve having a 
case reader, who had no counseling relationship with the student, 
predict educational achievement on the basis of all the informa- 
tion available up to, but not including, the counseling interview 
itself. Such predictions maybe compared with those made by the 
counselor after he interviews the student, Such crucial experi- 
ments are needed; a preliminary one will soon be reported by 
the authors. 

There are two assumptions that may be applied here as a 
basis for evaluating the counseling program. One objective that 
may be assumed for a counseling program is that of enabling 
students to compensate successfully for their disabilities in order 
to succeed. If that is an objective, then the expected evidence 
of efficiency in the counseling program would be a lower prog- 
nostic efficiency of a test battery for counseled students than for 
non-counseled students. If counseling is effective in this sonHe, 
then students who, if left alone, would fail, may succeed. 

Another objective of counseling may be to bring all factors 
other than those of aptitude (interest, opportunity, working con- 
ditions and so on) to a common level. Thus the performance of 
the students would be distributed according to their levels of 
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ability. The greater the excess o£ predictive accuracy for coun- 
seled over non-counseled, the closer the counseling program will 
be presumed to have come to the ideal — that of removing all 
influences other than ability which interfere with student 
achievement. 

Composite Criteria. All of the criteria discussed above have 
been partial criteria, since none of them was assumed to be 
evaluating all the possible objectives of counseling. We turn 
now to possible methods by means of which a more comprehensive 
evaluation of counseling may be secured. 

The Use of a Judgment Criterion 

It is at this point that a clear schism appears between an 
approach which is narrowly statistical and an approach which 
makes use of statistical methods in conjunction with the experi- 
mental situation. The former point of view has the desirable 
objective of clear-cut results, but, in its blind adherence to tradi- 
tional method, produces results which are unlikely to be significant 
either statistically or socially. This method would mechanically pool 
all of the part-criteria either in some form of average or in a profile. 
The method of averages compounds the artificiality which previ- 
ously had been indicated as inherent in the use of the part- 
criteria without reference to the individuality of each student, 
The method of profiles suffers from a lack of well developed 
statistical techniques for handling that type of data and, more 
seriously, from the fact that artificial data cannot be refined and 
validated by casting them into profile form. 

Rather than sacrifice meaningfulness for neatness of statis- 
tical treatment, the other approach has clearly recognized the 
impracticability, at the present, of getting more than rough 
measures of the general efficiency of counseling (33, 37). It has 
therefore attempted to use a judgment criterion by means of 
which the adjustment of the student is estimated in terms of his 
original problems and any of the available data, including the part 
criteria (16, 22, 27, 30, 31, 36, 38, 42, 43 : chap. IX). 

As described in Williamson and Darley the judgment of ad- 
justment is based upon a follow-up interview of the student (43: 
chap. IX). The status of the case at the time of follow-up is 
always considered in the light of the diagnosis and prognosis 
made earlier by the counselor. All of the various typcvS of data 
—grade achievement, the student’s statement of satisfaction and 
adjustment with regard to vocational orientation and choice, 
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information concerning the student’s activities, judgment of gen- 
eral attitude, etc.—are weighed by the judge according to their 
relevance to the individual case. In this way the possible errors 
inherent in a non-personalistic interpretation of objective data 
are minimized. 

The simplest experimental design requires that either the 
counselor or a case reader make the judgment as to the degree of 
the student’s adjustment subsequent to counseling and in contrast 
with his pre-counseling adjustment. A detailed manual of direc- 
tions, including examples of degrees of adjustment, is necessary 
in such an experiment (43: chap. IX). The results are reported 
in terms of the percentage of students counseled who achieved 
various degrees of adjustment. If the counselor makes such a 
judgment, it should be pointed out that, if he is a good one, he 
will know subtle angles and attitudes which are unlikely to be 
explicitly stated in the case records and which would be over- 
looked by an independent case reader. Many of the subtle influ- 
ences in a case may even exist in unverbalized form for the coun- 
selor and have no possibility of appearing in the case record. 
In addition, the counselor knows, perhaps better than anyone 
else possibly can, what he has been trying to do. The disad- 
vantages of using the counselor’s judgment lie first of all in his 
special interest in the results which may lead to an approach 
which is either too self-critical or too self -lenient; and secondly, 
in the undesirable consequence that the counselor's effectiveness 
in counseling may be decreased because of his awareness of hi.s 
responsibility for evaluating his own efforts. 

While the use of the independent case reader obviates the 


possibility of impairing the counselor’s effectiveness and intro- 
duces a theoretically impartial evaluator, it also has its draw- 
backs. As has been indicated, the case reader may miss many of 
the nuances. There are also so many conflicting philosophies and 
procedures and techniques in counseling that the case reader may 
be either unsympathetic with or ignorant of the counselor's spe- 
cific objectives. In order to achieve greater impartiality and objec- 
tivity, two case readers and an arbitrator have been used, Wil- 
liamson has reported the use of this method with three trained 
workers who had nothing to do with the diagnosis and' counsel- 
mg, but who collected data directly from the students for in- 
dependent and pooled judgments of effectiveness (42). With 
trained judges heterogeneity of point of view need not inter- 
fere with consistency in judgments. 
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While the experimental method outlined may yield evidence 
of the degree of improvement in the counseled population, it 
does not prove that cooperation with the counselor’s suggestions 
was a necessary condition. Evidence for the latter may be ob- 
tained by estimating the degree of cooperation of each student by 
one of the methods previously discussed. By comparing the ad- 
justment achieved by students who cooperated, with the adjust- 
ment of those who did not, we can determine whether cooperation 
was necessary and to what degree* If we find that those who 
cooperate adjust better than those who do not, we still have a 
question of whether the degree of adjustment achieved by those 
who did not cooperate might not be equalled or bettered by those 
who received no counseling at all. A matched non-counseled 
group would seem to be the only means of providing an answer 
to this question. We have already discussed the possibilities of 
the matching process. If we were to attempt to avoid the 
matching problem by counseling every other student who comes 
for counseling, retaining the other half of the group as controls, 
we would be doing violence to a social canon. The real solution 
must await a time when we have sufficiently isolated treatment 
techniques and problems to compare two treatments used with 
the same type of counseling problem. 

General Considerations 

Our consideration of the types of criteria and the methods of 
measuring them, feasible in the evaluation of the effectiveness 
of counseling, has touched upon definite limitations on exact 
evaluation of counseling. Whether these weaknesses will be 
insurmountable and will restrict evaluation to rough, rule-of- 
thumb methods depends upon future progress in experimentation. 

One type of difficulty is the inability to set up clear delinea- 
tions of the problems and variables involved* This has been 
traced first to the inadequacy of descriptions of diagnostic and 
treatment techniques of the counselor plus the gaps in knowledge 
of student problems (40: chap. XXVII). A second source is 
the element of uniqueness in the student’s problems and the coun- 
seling techniques appropriate for them. The criteria which have 
been considered have the weakness of being either too gross a 
measurement or so far removed from the individual as to lack the 
quality of meaningfulness. If, in the future, methods are devised 
for providing more adequate criteria, then more exact experi- 
mentation may be made. 
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A second limitation, inherent m the nature of the counseling 
situation, is the sampling difficulties involved in setting up con- 
trols. This difficulty has been stated by Murphy et aL. ‘Tn con- 
nection with the control group, there should be noted the relative 
impossibility of securing a true 'control/ The fact that the ex- 
perimental group applied voluntarily for counseling introduces 
a selective factor unmatchable among non-clients” (23: p. 952), 
Although this assumption has never been established even in the 
plethora of learning experiments, only assumed, caution should 
be applied in interpreting the results of matching experiments. 
We also lack adequate techniques for matching students for such 
pertinent factors as interest, perseverence and other similar qual- 
ities. At the same time, it is impossible to set up an experiment 
which would entail selecting cases from the general population, 
since willingness to be counseled would seem to be one of the 
necessary conditions for counseling. These limitations in sam- 
pling methods imply that evaluation must necessarily be a long- 
time process, involving a great deal of experimentation with 
different methods. 


The condition that diagnosis and counseling cannot be 
studied separately is a further complicating factor. When the 
counselor has made a diagnosis of the student’s problems, its 
causes, and the types of treatments that are likely to solve it, 
he cannot determine whether his diagnosis was correct unless the 
student carries out the recommendations. For example, if a 
counselor's diagnosis states that student A can do effective* work 
in college only by following certain of his recommendations, 
student A must remain in college for that diagnosis to be tested. 
The inability to control the conditions necessary for an adequate 
tryout of counseling recommendations often precludes determina- 
tion of the effectiveness of the advice. Factors which are often 
beyond the control of either the counselor or the student include 
restricnon imposed by the school administration, those imposed 
by social codes, prejudices and attitudes of students or parents 
and lack of proper placement facilities. 

aonUcation1or^^'^ discussed in this paper have little 

ILl T ^ comparison of individualized counseling with 
other types e.g.. group, traditional, casual interview etc This 
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used by the counselor. Grade or information achievement repre- 
sents the only type of criterion that can be applied in evaluating 
this type of counseling. 

Our discussion has shown that there is a need for more sys- 
tematic studies, using the more feasible part-criteria. Other 
approaches have been indicated as having possibilities. The rela- 
tionship between the student*s educational and vocational objec- 
tives and his level of ability should be studied further as a 
criterion of adjustment. Some studies have already yielded some 
preliminary results (1, 10, 29, 32). In the last few years this 
problem has been receiving attention under the term level of 
aspiration. Studies have revealed some provocative principles 
under laboratory conditions (11, 12), but we must learn whether 
these principles have validity for life situations. We should 
determine whether success in one area, i,e., vocational or educa- 
tional, has an effect on the level of aspiration in other areas of 
adjustment. What are the relations between level of aspiration 
and feelings of failure? Can vocational or educational success 
be such a potent factor that it would outweigh other experiences 
in determining an individual’s general success-failure feelings? 
How vital are social group factors in determining the individ- 
ual’s level of aspiration? To what degree do levels of aspiration 
persevere at various age levels? The answers to these questions 
would seem to be pregnant with implications for both the coun- 
selor and the evaluator. 

If and when our knowledge of student problems and of diag- 
nostic and treatment techniques has advanced sufficiently, we will 
have the opportunity to carry out more exact investigations. At 
this point, we can foresee experimental designs which should 
be applicable when such advances are made. One possibility is 
an experiment in which individuals having problem A will be 
divided into two groups, one which will receive treatment li the 
other treatment 2. In this way we may determine which specific 
techniques are most effective for a particular problem. 

Another plan of experiment could be designed to determine 
for what types of problems a treatment is applicable, Here, two 
groups, one representing problem A, the other problem B, would 
both receive treatment 1, Both methods could be expanded to 
include all types of treatments and problems. Criticism of these 
designs may be directed at the apparent assumption that problems 
may appear isolatedly, That this is extremely unlikely cannot be 
denied. Yet, assuming advances in our techniques and proced- 
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ures, it seems possible that the types of factorial design used in 
analysis of variances can be utilized to take care of the effects 
of interactions among treatments for various problems. The 
success of such an experiment v/ill depend upon the discovery of 
a number of cases in which one problem is clearly present and 
other types are minimal in significance or complexity. 

Summary and Conclusions 

1. All available methods of evaluation have weaknesses. 

2. Composite criteria which avoid arithmetic combination of 
the part-criteria are at present least open to question, although 
still being crude measures. 

3. The problem of securing sufficient data without doing 
violence to the concept and practice of counseling is a real one. 
Involved also are the inadequacy and incompleteness of most 
available case records. 

4. The proper time interval to use for evaluation is extremely 
important because of the possible relationship between the inter- 
vention of confusing factors and the length of time between 
counseling and evaluation. 

5. The methods used for validation of diagnostic and prog- 
nostic tools (e.g., tests) may not be applicable because of the 
uniqueness of each counseling situation. Stated another way, the 
methods of studying students in general may not be applied to 
the study of individual students with particular problems. 

6. An impediment to more exact evaluation is the inability to 
control conditions for an adequate test of counseling recommen- 
dations. 
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THE LOGIC OF AGE SCALES* 

M. W. RICHARDSON 
United States Civil Service Commission 

An age scale is a type of psychological test designed to 
measure general mental ability ("intelligence") in terms of 
performance of various mental tasks found to be normal for 
various ages. The child ■whose performance i.s typical of ten- 
year-old children, for example, is said to have a mental age of 
ten, The method was invented by Alfred Binet, Although the 
techniques have been modified in detail by British and American 
psychologists and the device of the intelligence quotient (LQ.) 
has been appended, the main outlines of Binet’a work have been 
retained. The most widely used of the age scales is the Stan- 
ford Binet. 

The age scale has been widely accepted. The I.Q., in particu- 
lar, has passed into the language of the general public, together 
with the common misconception:; connected with its brief hi.story 
in science. The fact that a device has attained wide ti.sc is not a 
guarantee of its soundne.ss; and it is sometime.^ necessary in the 
interest of sound scientific advance to examine critically proce- 
dures and devices in common use. If the criticism.^ in this paper 
seem to be directed chiefly to one particular age scale, the expla- 
nation is that this one scale is the most widely used and has 
been most carefully constructed and standardized, 

A person whose academic specialty is the logic of science 
addressed a group of psychologists on the necessary conditions of 
measurement. He discussed the familiar matter of equality of 
units and the operational test of equality of units by the coinci- 
dence of any part of the scale with any other upon superlmposi- 
tion. He mentioned the matter of measuring a single variable at 
a time, and the necessity of having a real origin of measurement, 
if ratio comparisons are to be made. He followed this sound 
discussion of the scientific method with a curiously erroneous 
one; he congratulated psychologists in having, in the Binel age 
scales, a measuring device that meets all three of the requirements 
for a scientific measuring device. The writer and others carefully 

* This article is adapted from a chapter in a forthcoming book On test 
theory by the same author. 
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pointed out to the speaker that the age scale meets not one of the 
three requirements set up; it has no real origin of measurement; 
its units are not equal, and it does not isolate a single, unitary 
variable for measurement. 

When it is uncritically considered, the age scale idea seems 
to be a happy one. What could be more simple and direct than 
to see how high the child can ascend an evenly graded scale? 
The concept of normality of performance made it the most 
natural thing in the world to describe such test performance in 
terms of the age for which the performance was normal. The 
concept of the age scale is a deceptively simple one, however, and 
the writer is of the opinion that the unstated special requirements 
and limitations of the age technique have received too little atten- 
tion. It is true that during the twenty-one years between 1916 and 
1937 many papers pointing to difficulties in scaling, scoring, and 
interpreting the Stanford Binet appeared. Moreover, several 
issues were kept continually in the foreground of attention. 
Unfortunately, the distinction between purely psychometric issues 
and psychological issues was not always made. The result is that 
certain deficiencies and limitations belonging to the mechanics 
of test construction were misinterpreted as psychological issues. 
A case in point is the constancy of the I.Q., about which it will be 
necessary to say more later. 


Validity oi Age Scales 

A Binet scale consists of a series of sub-tests or items designed 
to measure '‘general intelligence,” whatever that may mean. For 
example, the 1937 edition of the Stanford Binet contains 127 sub- 
tests graded in difficulty from tests suitable for two-year-olds to 
those suitable for superior adults. The sub-tests were selected in 
the process of construction from a larger number of sub-tests 
It is pertinent to inquire into the method of selection of the sub- 
tests. In what respect does the method of selecting the sub- tests 
insure that the resulting scale will be valid? One of the devices is 
o plot the percentage of correct responses to any given sub-test 
against the chronological age, after the sub-test has-been apnlied 
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of the Binet tests many compromises with practical expediency 
are made. Under special conditions a mathematical function can 
be used to describe a discrimination curve of items against chron- 
ological age. It has been shown that the use of this function is 
simply an alternative to the correlation methods, and precisely 
equivalent to them under certain special conditions. 

This type of analysis throws light on the 'Validating” proce- 
dure. The retention of sub-tests on the basis of sharp curves of 
age discrimination is the same as retaining items that have the 
large correlations with chronological age. The criterion of valid- 
ity is simply chronological age, and the practical effect of the 
procedure is to select items that have relatively high correlations 
with chronological age. 

The procedure leads to a serious difficulty. The standing high 
jump, or other athletic skills, yield similar discrimination func- 
tions, since they are likewise positively correlated with chrono- 
logical age. The method of item selection thus breaks down as a 
way of attaining validity. The only criterion of validity remain- 
ing is the judgment of the persons constructing the scale. An 
allied consideration is that the selection of items on the basis of 
high correlation with mental age on the same or previous scale, 
is merely a measure of internal consistency or reliability. Nothing 
in the general procedure operates towards the selection of items 
that measure a unique trait. An interesting logical difficulty 
appears. Suppose that, out of the hand-picked collection of items 
supposedly measuring the aspects of intelligence desired in the 
scale, the items selected are those which have the steepest dis- 
crimination functions. Let us assume further that two items are 
so discriminating and so far apart in proper agedocation that 
their discrimination functions do not overlap. The result is that 
the two hypothetical "good” items or sub-tests have a iscro corre- 
lation, A scale made up of such items must necessarily be unreli- 
able as a composite measure. Furthermore, to the extent to which 
the search for valid sub-tests by this procedure should be success- 
ful, the number of different factors measured would increase. 
Evidence at present suggests that no fewer than six different 
mental functions are measured in a higgledy-piggledy fashion 
by the Stanford Binct. The multiplicity of factors is perhaps not 
so serious as the fact that different things arc measured at differ- 
ent ages. The sobering fact about the age-acale technique is that 
we do not know what is being measured, or what any given inteIH« 
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gen.ce quotient means in terms of the relative standing of the 
individual. 

As an added cornplicatLon^ the score received by the testee is 
expressed in terms of jnsntBl Mental a^cs are statistical 

numbers based on the concept of normal or average test perform- 
ance of children of a given age, when the sample of children is 
representative of some population of children. On page 25 of 
Measuring Intelligence, Terman and Merrill state that the expres- 
sion of a test result in terms of age noi ms rests upon no statistical 
assumptions. The statement is erroneous and misleading, The 
truth of the matter is that the mental age is a measure derived 
from raw scores in accordance with certain assumptions; it is as 
definitely statistical in nature as the standard score, for example. 

In using scales of the Binet type, we choose to express test 
performance, not in the arbitrary units of number of items passed, 
but in terms of “mental years and months.’* The raw score ^'units'* 
are of course arbitrary, in the sense that they are not units at all. 
The child who answers 12 items correctly cannot be said to exceed 
the child who passes 9 by the same amount that the latter sur- 
passes a third child who answers 6 correctly. No ordinary test 
can be expected to satisfy the additive property required for 
measurement on a scale. But the mental year or the mental month 
is likewise not a real unit of measurement. In order for the 
mental year (or month) to be a real unit of measurement, it 
would be necessary for the function representing mental growth 
to increase regularly with chronological age. If, during each 
year, a child had the same increment of mental growth, the men- 
tal year or mental month would be constant in value. Plowever, 
it is commonly agreed that the child matures less and less rapidly 
as he grows older, in intelligence as well as in physical character- 
istics, The annual increment of “intelligence/* i.e., a mental year, 
steadily becomes less until mental maturity is reached, at which 
time it is zero. An age scale is, therefore, not a true scale because 
it is not built up from equal units. In this connection, it may be 
noted that the true shape of the mental growth curve cannot be 
determined from scores expressed in terms of mental ages. If 
such were attempted, one would get results predetermined by 
the crude growth curve adopted in order to express raw scores 
in terms of mental ages. 

Whatever the merits of the assumed growth curve may be, the 
crucial consideration is that its true shape and its upper limit 
cannot be determined by use of a “scale** expressed in terms of 
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mental ages. The true shape of the mental growth function can 
be determined, strictly speaking, only by use of a scale with real 
units of measurement. Once the mental growth function is estab- 
lished, it is possible to calibrate the underlying true scale in 
terms of mental ages, if desired. Then the interval between the 
mental age qf four and the mental age of five might be cxprevssed 
as a certain fraction of the scalar unit, the mental year between 
five and six as a somewhat smaller fraction of the same real unit, 
etc. Finally a place would be reached where the mental year is 
a negligible fraction of the real unit, and therefore has a value 
of practically zero. We would then have a proper (although 
indirect) experimental solution of the problem of the limit of 
mental maturity. The widespread use of mental ages has not 
helped to solve the problem, mainly because the use of mental 
ages as derived measures begs the question* 

Although the exclusive use of mental ages forever begs the 
question of the limits of mental maturity, it is urged that a 
definite and well-accepted social meaning has been attached to 
them. It seems simple enough to define mental age as the average 
or median test performance of typical nine-year-old children. 
The definition works well enough until the limit of mental matur- 
ity is reached. If the limit of maturity is assumed to be 15, a 
mental age of more than 15 is impossible, by definition. Mental 
ages of more than 15 are assigned in the process of standardiza- 
tion to test performances by use of *‘cut-and-try‘' procedures 
based on some not well-defined assumptions. At best it is unfor- 
tunate that the definition of mental age must be radically sliiftcd 
at one point or region in the age scale. 

The L Q, and Its Troubles 

To multiply confusion, the device known as the intelligence 
quotient has been adopted. The intelligence quotient is defined as 
100 times the ratio of mental age to chronological age, and is thus 
an index of brightness. An index of brightness can of course 
be no more than a statistic relating the individual's test perform- 
ance to the average performance of those of the same age. It is 
exactly as true of I.Q. as it is of other possible statistics serving 
the same purpose that one must always use it in connection with 
some measure of variability of test performance within the age 
group. Obviously one measure taken from a distribution has no 
meaning unless a measure of dispersion is given. It might be 
argued that clinicians keep in mind some kind of subjective scale 
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which serves in lieu of the ordinary statistical parameters. If so, 
the mental feat is remarkable since the various age groups, in one 
age scale at least, have dispersions of intelligence quotients which 
vary considerably. The difficulty of interpretation of the single 
statistic (the I.Q.) is greatly increased where the standard devi- 
ation of I.Q.’s of one chronological age group may be twice as 
large as that of another age group. The intelligence quotient 
shares with the mental age from which it is derived the fictitious 
character of measures above the maturity level. It is nonsense 
to describe an adult as having an I.Q. of 120 because such a state- 
ment is based on an irrational definition and is unverifiable exper- 
imentally. It is the practice of Binet testers to assume some defi- 
nite chronological age as the upper limit of mental growth. Thus, 
it is assumed on at least two age scales that the upper limit of 
the average child is reached at the age of fifteen. The crucial 
difficulty in making such an assumption is not that the upper 
level set may be wrong, but it lies in the utter impossibility of 
checking up on its correctness by means of age scales, 

One of the moot questions about the I.Q, is its constancy. 
It seems unfortunate to the writer that so much time of psycholo- 
gists has been wasted on such a matter. It seems that what ought 
rightly to be merely a formal problem in test construction has 
been translated into one of spurious psychological significance. 
The only question properly asked at this time may be definitely 
stated: Did the authors of the age scale succeed in constructing 
a device which gives a constant I.Q.? Questions involving changes 
in I.Q. possibly attributable to environmental factors must always 
take into account the fluctuations of the I.Q. which are due to the 
test and to the statistical operations used to determine the intelli- 
gence quotient. Failure to consider the expected magnitude of 
fluctuation of the I.Q. may easily result in gross misinterpreta- 
tions of time-changes in its value for any one individual. 

Before we can properly evaluate the effect of organic and 
environmental factors on the intelligence quotient, we are forced 
to consider the variations in the I.Q. inherent in the testing 
technique.^ The fluctuations are those associated with the concept 
of reliability. For the 1937 Revision of the Stanford-Binet Scale 
the estimated reliability coefficient varies from 0.90 to 0.98, the 
higher reliability being associated with the lower I.Q, intervals 
A median value for those near 100 LQ. is 0.92. A representative 
value of the standard error of measurement is 4.5. Certain sys- 
tematic variations in the individual I.Q. are also found. The 
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practice effect is one type of systematic variation. Terman and 
Merrill estimate that the mean increase in I.Q. on the second test 
(which means on the other form, since two forms, L and M, are 
provided) ranges from 2. to 4.4, when the time interval between 
testings is short. The increase due to practice effect is presum- 
ably greater when the same form is repeated. 

Another systematic source of error may lie in details of the 
construction of the age scale. Thus, it might be a characteristic 
of certain age scales that the I.Q, of a superior child decreases 
with chronological age. Strictly speaking, nothing in the defini- 
tion of the I.Q, requires constancy during the entire period of 
development. The constancy of the I.Q., if it exists, is imposed 
by the process of standardization. Therefore, an experimentally 
obtained constancy of the I.Q. proves only that the scale has been 
constructed in such fashion as to produce constant I.Q.^s except, 
of course, that random fluctuations will still be present. When, 
and only when, an age scale has the characteristic that I.Q/s of 
individuals at all age levels tend to remain constant, one may 
attach significance to the case of the unusual individual whose 
I.Q. does not remain constant. The significance of any such shift 
of I.Q. in time must be judged in relation to the normal shift to 
be expected by random error, or unreliability. 

Increases or decreases of the order of magnitude of three 
times the standard error of measurement must first be tested with 
respect to the magnitude of variable errors of measurement before 
it is legitimate to entertain the hypothesis that some other factor 
such as special therapy, change in environment, or organic change 
is responsible for the shift in I.Q. In addition, obtained increases 
should be scrutinized carefully from the standpoint of possible 
practice effect. All such interpretation is predicated on the basis 
of constant I.Q., as built into the scale itself. One may properly 
inquire just how a scale with constant I.Q. may be constructed. 
In view of the inherent difficulties with the mental age and intel- 
ligence quotient, it is impossible to state any perfectly general 
rules. However, the part of the scale which is treated as if the 
growth curve were linear, viz. two to 13 years, can be abstracted 
for discussion. The problem the test constructor faces is that of 
providing that most individuals will be assigned the same I.Q., 
within the reliability of the scale, every year from two to 12 
inclusive. If the various sub-tests are properly scaled, i.e,, 
assigned to a given year level as a median performance of unse- 
lected children of that age, the I.Q. of 100 will remain constant 
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Inconstancy is to be expected in intelligence quotients other 
than 100, unless certain conditions are satisfied. One condition is 
that the variability of test performance increases for successive 
age groups from two to 12. If we assume that children of a 
given typical group vary more among themselves as they grow 
older, it is possible to arrange matters so that the I.Q. of most 
individuals will be constant. Suppose that we have an individual 
whose I.Q. at age eight is 120. The mental age from which the 
I.Q. is estimated is 9.6 years (or 9 years, 7 months, approxi- 
mately). Let us further suppose that we have a distribution of 
mental ages of all the children in our sample of eight-year-olds. 
The mental age of 9.6 years is, say, one standard deviation above 
the mean of the distribution of eight-year-olds. The standard 
deviation of the distribution is 9.6 — 8 — 1.6 mental years. 

Now, if the same child is to have an I,Q, of 120 at the age of 
nine, his mental age will then be 10.8 years. If the nine-year-old 
sample is composed of the same children we should expect, except 
for errors in measurement, that the child will have the same posi- 
tion in the nine-year distribution that he had in the distribution 
of eight-year-olds. Since his mental age is now 10.8, the standard 
deviation of the distribution of nine-year-olds is 1.8 mental year. 
Similarly, the standard deviation of the ten-year-olds must be 
2.0 mental years; of eleven-year-olds, 2.2 mental years; etc. The 
preceding illustration shows that, for an assumed linear growth 
function, the standard deviations of the mental ages must have 
constant increments for each advancing year. How shall this he 
done? Considering, for sake of simplicity, that we have just six 
sub-tests at each year level, we may increase the standard devi- 
ations of successive year levels by (a) selecting sub-tests which 
have higher mtercorrelations at the older age levels, (b) assign- 
ing a larger number of mental months to each sub-test. It will be 
seen at once that the latter is inadmissible since a total of 12 
mental months is assigned at each year level. The conclusion is 
inescapable that the degree of correlation between sub-tests must 
increase steadily with higher age levels if the I.Q fs to be 

thJ^i! treatment is theoretical and does not imply 

tnat the authors of anv < * X 

32 



THE LOGIC OF AGE SCALES 

dictable fashion; it may be sufficient to account for the relative 
constancy of I.Q/s attained in age scales. 

It is difficult, however, to account for the attainment of con- 
stancy of I.Q. and, at the same time, approximately equal disper- 
sion at the various age levels. The range of standard deviations 
of I.Q.’s reported for various half-year levels is from 12,5 to 20.7, 
A representative value is 17. The values vary consideiably, prob- 
ably because of accidents of scale construction. Certainly the 
values given by Terman and Merrill do not vary systematically 
with age, and the authors assume that the true variability is 
nearly constant from age to age. However, let us consider a 
half-year group as an approximation to ''point** age. If all indi- 
viduals within such a half-year group are considered to be of the 
same chronological age, the mental ages are proportional to the 
intelligence quotients, i.e,, a plot of M.A. against I.Q. is linear. 
Even for a half-year interval, approximate linearity mu.st hold; 
otherwise the definition of an I.Q. is meaningless. 

It follows that if half-year groups have the same I.Q. disper- 
sion, they must have approximately the same mental age disper- 
sion. But the mental age dispersions must increase from year to 
year in order for I.Q.*s of individuals to be constant. It thus 
appears that two possible properties of the I.Q. are inconsistent 
and not attainable at the same time, in any strict sense. The most 
serious criticism to be directed against Terman and Merrill’s dis- 
cussion of the matter is that they tend to treat the (roughly) 
approximate equality of dispersion of I.Q. at the various age 
levels as experimental facts, as perhaps having psychological sig- 
nificance, The I.Q. is a statistical concept, having the properties 
we put into it by the accidents of cut-and-lry scale construction 
or which we force it to have by conscious dCvsign. If we postulate 
that our statistical index shall have certain properties, we 
can then construct a test in accordance with our imposed 
requirements. 

The only reservation is that we may possibly have imposed 
characteristics which are mutually inconsistent, in which case 
we perforce discover the source of the difficulty. If we fail to 
limit the properties of a statistic by rational design, the vagaries 
of that statistic will be brought to light in subsequent empirical 
studies. The result is the raising of such false issues as the 
constancy of the I.Q, The gist of the matter is that the I.Q. 
can be made to be constant, if that is thought to be a desirable 
property. If the scale is not constructed in such a way as to give 
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constant I.Q.’s for (most) individuals, the die is cast and the 
l.Q. will not be constant. 

In summary, it may be stated that the age scale technicjue » 

(1) possesses no advantage over group test methods 

(2) has no straightforward rationale so that the process 
of standardization may proceed without the necessity 
for ^^adjustments^’ 

(3) meets none of the three requirements of real mental 
measurement 

(4) leads to much useless work in correcting the previous 
^^standardization” 

(5) embodies the mental age and intelligence quotient, both 
extremely unfortunate concepts 

(6) leads to problems of spurious psychological signifi- 
cance, such as the constancy of the l.Q. 

(7) makes impossible any solution of the mental growth 
function 

(8) has led to dubious devices and untenable interpreta- 
tions of various sorts, among them “scatter” and meas- 
ure of “mental deterioration ” 

It is recommended that the age scale technique in its present 
form be abolished in its entirety, and that it be supplanted by 
reliable homogeneous group tests of single functions. The latter 
can be recombined, if desired, into a single index of mental 
capacity based on position in year group, A better procedure is 
to continue work towards some real unit of measurement, to the 
end that departures from normal growth in several functions may 
be discovered and clinically interpreted. If. by reason of demand 
from teacher, parent, or psychiatrist it will seem necessary to 
give a general index of (average) mental level reached, it can be 
done by use of a suitable combination of measures furnished by 
the separate tests. It is desirable, however, to avoid the use of a 
single index of mental level. 

It has been urged in defense of the Binet test that during its 
administration, the trained clinical psychologist has an opportu- 
nity to make observations of the child’s behavior other than that 
required for rating “general intelligence.” It is maintained that 
such observations may have as much value as, or more value than, 
the mental age, in getting a **clear picture of the individual 
tested. If such clinical insights can be reliably obtained and 
recorded, the obvious desideratum is a standardized interviewing 
technique, to be applied and interpreted entirely separately from 
the measures of primary abilities. 
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COUNSELING ON THE BASIS OF INTEREST 
MEASUREMENT-^ 

JOHN G. DARLEY 
University of Minnesota 

As the counselor studies his available data on abilities, 
achievement, interests, personality, and background o£ the Stu- 
dent facing him in the intervievir, he must select a conversational 
starting point that will establish rapport and get the interview 
under way. At some early time he must discuss the student's 
stated reason for seeking help, and eventually he must interpret 
the interest test data in a manner understandable to the student. 
Assume that the student makes A scores on the occupational keys 
for Y.M.C.A. secretary, and personnel manager, and B J for 
school superintendent and social science teacher on the Strong 
Vocational Interest Blank, Assume that his claimed occupational 
choices are business, engineering, and "executive work.” He feels 
the need of help in making a final occupational choice. 

At the point of interest test interpretation, the counselor can 
make this bald statement: “You have the interests of a Y.M.C.A. 
secretary or a personnel manager!" With minor modification.H 
this is probably the standard approach to interpretation. There 
is no more probable way to lose a case than this. It is the lea.st 
effective clinical approach, for the following reasons : 

1. The student’s spoken or unspoken response is usually 
“How can you say that? I never was a Y.M.C.A. secretary 
or a personnel manager!” At this point the counselor must 
backtrack and start a rather incoherent explanation of the 
basis of interest measurement, to his own and the student’s 
confusion. 

2. If the student accepts the statement without raising 
the foregoing issue in some form, the chances are he will 
re-mterpret the statement, then or later, to mean that he 
has the ability to be a Y.M.C.A. secretary or a personnel 
manager, and that these arc two jobs where his success is 
guaranteed. If any other factors interfere with curricular 

*This article is the first draft of a chapter in a forthcoming mono- 
graph entitled: Clinical Aspects and Interpretation of the Strong Voca- 
tional Interest Blanks. Other thcoictical and intei'(irclive phases of inter- 
est measurement are treated more extensively in the monograph, to he 
published by the Psychological Corporation. 
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or job success, he claims he “was told he would succeed in 
these jobs.” 

3. Such statements run the risk of flouting countless 
stereotypes, prejudices, specific dislikes, or misconceptions 
evoked by occupational labels, either in the student or in 
his parents. Very few people know what a personnel man- 
ager does, and there are substantial and not always com- 
plimentary stereotypes about Y.M.C.A. workers. The in- 
teresting fact that these labels are directly at variance with 
the student^s claimed choices operates also to set up re- 
sistances in the student, although the student^b own specific 
choices are also hedged around with favorable stereotypes 
which may be equally invalid, and although there is con- 
siderable evidence on the instability and invalidity of the 
student’s claimed choices. 


4. Such statements run the risk of moving the discus- 
sion too early in counseling to the temporarily irrelevant 
factors of opportunities, salaries, prestige values. The 
counselor is forced to waste precious time giving data (if 
he knows of any) on these points before having cstablislied 
an understanding of the interest type being discussed. 

5. Such statements fail to take into account the vital 
factors of levels of ability and past achievement, which 
determine the level of future academic achievement most 
probably attainable; educational disabilities affecting edu- 
cational progress in a conect curricular and occupational 
area; amounts of relevant specific aptitudes, in additiau to 
level of general scholastic ability; and personality char- 
acteristics related to job success or satisfaction. Specific 
patterns of interest unaccompanied by ability and past 
achievement sufficient to permit curricular competition in 
professional schools occur frequently in counseling, be- 
cause of the relatively low general correlations between 
measured interests and measured abilities or achievement. 

trong has published correlations of each occupational 
key with an intelligence test.i On the original blank the 

Brmtle collected interest test scores, college grades and 

uiiliTi’ifr* ‘^allege® freshmen. 

,.® interest test scores for the keys for doctor, lawyer 

manager, and purchasing 
agent, they found only one positive correlation above 40 
with selected parts of the Iowa High School Content Ex- 
mination— the correlation between engineering interests 

est ^com as^MeasuKd^by^ the”stcj**“ Occupational Inter- 

Teat Seaulta and College ^Marfca in Achievement 
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and measured achievement in mathematics. Achievement 
in mathematics and science correlated .28 and .29, respec- 
tively with measured interests in medicine. Achievement 
in English literature, science, and social studies correlated 
— .43, — .26, and — .26 respectively with measured interests 
of a purchasing agent. The correlations between subject 
matter grades and measured interests weie even lower than 
those between achievement tests and meavsured interests. 
Grades in mathematics and science correlated only to the 
extent of .14 with interests in engineering, while grades in 
history correlated — .47 with interests m engineering. The 
authors were sufficiently encouraged by these relations be- 
tween scholastic accomplishment and interest test scores 
derived from studying adult occupational groups to .suggest 
that ‘^scales for scoring the Strong Interest Tests should 
be devised for the principal subject groups in higher sec- 
ondary education.” However, the obtained correlations were 
so low that the clinician must be extremely careful to keep 
interests and abilities or achievement separate in his own 
thinking, and to see that there is no such confusion in the 
student’s thinking. 

This error in counseling is particularly tragic and in- 
excusable where the occupations being discussed in terms 
of the interest test are those for which society demands 
college training prior to certification for professional com- 
petition. It is equally inexcusable in case.s where the occu- 
pation can be entered with or without specific advanced 
training, as in the case of general measured interests m 
business. But in such cases, the counselor can cover his 
error by saying later what he should have explained earlier, 
namely, that in such occupaliouvS, vsuccess or satisfaction 
in the occupation is still possible even though success or 
satisfaction is not possible in a curriculum which may bear 
some degree of resemblance to the occupation, but which 
is not yet an indispensable prerequisite. This explanatory 
technique can be effectively used in “downgrading” some 
cases. 

6. Such statements also fail to take into account the 
problem mentioned earlier^' m regard to the present-day 
representativeness of norm groups, as exemplified in the 
psychologists’ key. 

7. Finally, such blunt statements omit consideration of 
possible changes of specific measured interests which, while 
infrequent, may occur under certain conditions. Strong 
states this position clearly: “Prognostication of future be- 
havior cannot safely be based upon the presence or absence 
of any single interest, but it does appear that to a consid- 

The monograph from which this chapter is taken discusses the repre* 
sentativeness of Strong's standardizing groups as a factor in interpreta- 
tion of the interest test results. 
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erable degree at least it caa be based upon the entire con- 
stellation of interests/’*^ In the article quoted, test-retest 
correlations of the specific keys ranged from ,59 to .84 over 
a five-year interval beginning with the senior year in coU 
lege. 

Furthermore, in using the blank with younger btudenta, 
it is usually more important to determine the interest typ« 
than the specific occupational interest Carter, Pyles, and 
BretnalP have demonstrated the presence of the types at 
the average age of 16.5, whereas Carter and Jones‘s have 
shown that only 17 per cent of tenth-grade students receive 
specific A scores on the keys appropriate to their occupa- 
tional choices. Thus the counselor who uses the test with 
younger cases must remember that the standardising pro- 
cedure based on levels of scores made by adults may not 
yield an A score to a high-school student on a key within 
the interest type in which he may have a legitimate and 
dominant pattern. 

With this understood, the test becomes clinically useful 
in the age range from about 15 yeais and up. But the 
counselor who looks only for single A scores cannot make 
effective use of the test in this age range. This difficulty 
would be clearly eliminated if a technique such as standard 
scores could be used as the reporting device for younger 
cases. Then the higher pattern of standard scores within 
an occupational group would stand out more clearly on the 
individual’s profile, where the letter grade scores, based on 
adult norms, do not show intra-individual patterns so 
clearly in younger cases, 

These statements of the ineffective way to interpret interest 
test scores, and the reasons therefore, grow out of bitter clinical 
experience. There is fortunately a more effective alternative. 
Suppose, in this hypothetical case, no reference is made to the in- 
terest test scores until late in the counseling interview. Suppose, 
further, that the counselor draws out of the student, by question- 
ing, the reasons behind the student’s own choices of business, 
engineering, or executive work.” He will discover much super- 
ficial thinking about jobs, which is in itself important. But he will 
also_ discover the specific factors leading to the choices; infor- 
mation (or misinformation) regarding salary scales and “over- 


at an'i E. P. Bretnall, "A Comparative Study 
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crowded’* or ''undercrowded” fields and job duties; satisfaction 
expected from the job; self-estimates of strong and weak abilities 
or subject-matter fields; evidences of family pressures or tradi- 
tions dictating the choices; self-estimates of aspirations and mo- 
tives that are operative in the choices; and evidences of out-of- 
school experiences shaping the choices. 

Suppose, finally, that the counselor is familiar with the “in- 
terest types” or “interest patterns” growing out of factor analysis 
studies.'^ The counselor can then direct the questioning at get- 
ting the student to evaluate activities which are related to the 
interest type and which are within the scope of his experience 
with his environment. Questions can also be used to evaluate 
those experiences contra-indicating the type into which the stu- 
dent's claimed choices fall. 

Specifically, in the hypothetical case, unhappy experiences 
with mathematics would contra-indicate the technological inter- 
est type, in which the claimed choice of engineering is included. 
Participation in Hi-Y work and summer camp jobs may be drawn 
out as bits of evidence in favor of the welfare or uplift type in 
which some of the weasuzed interests fall. A discussion of “exec- 
utive work” as a pervasive problem of dealing with people takes 
It out of the claimed realm of a business activity alone. 

Notice that the student has not yet been informed of his own 
specific measured interests. Notice also that the counselor has 
used the test scores in directing his questions to evoke relevant 
experiences and to clarify the student’s thinking about jobs. At 
or near this point, the counselor will be ready to tell the student 
what his basic interest type seems to be, with some chance of 
getting this idea across by saying: “It seems to me that your 
basic interests are in helping people or in working with them 

'^Available factor analysis studies establish the qualitatively different 
types of interest patterns somewhat as fellows: interest in scientific or 
technological activities; interest m verbal or linguistic activities; interest 
in business contact activities; interest in. business detail activities; inter- 
est in welfare or uplift activities. The specific occupational keys for 
which the men’s interest test is scored may be approximately grouped m 
these five categories. To make a clinical determination of the intensity of 
the interest type, the following procedure has been used with experimental 
verification, including tabulations of frequency of occurrence: for an indi- 
vidual student, the pr/mary interent pattern is the interest type within 
which he shows a preponderance (majority or plurality) of A and B + 
scores on the specific occupational keys; the secondary interest pattern is 
the interest type within which he shows a preponderance of B-(- and B 
scores; and the tertiary interest pattern is the interest type within which 
he shows a preponderance of B and scores on the specific keys. 
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in an effort to bring about an improved adjustment, rather than 
in technical, impersonal activities, or in piling up a tremendous 
fortune/’ Then he may discuss specific occupational duties and 
labels as representatives of the basic type, phrasing his remarks 
somewhat as follows: ''These basic interests in helping people 
(or working with people) would be satisfied by the job of a per- 
sonnel manager, for example, who is responsible for . . , (and 
then may follow a description of job duties and tcsponsibilitics 
and types of training) . . , . ; or those same interests would find 
an outlet in the type of work that a Y. secretary might do. So 
far as training is concerned, these two jobs require somewhat 
different types of abilities and aptitudes as we can see in studying 
the two curricula involved; therefore, it is important to see how 
your abilities and past achievements line up with the two 
choices . . . . ” 

In this way the A and scores are introduced as examples 
of occupational outlets for the interest type rather than rigid 
occupational prescriptions for this student, and due allowance 
can be made for existing curricular differences. 

The advantages of this clinical procedure are obvious. It 
reduces to a minimum the arousal of resistances growing out of 
stereotypes or prejudices which the student may have about the 
occupational label. It permits the counselorj subject to his own 
imagination and knowledge of jobs, to generalise beyond the 
available keys on the blank and classify other occupations within 
the basic interest type, which is valuable when one rcalisses that 
there are about 20,000 occupational labels and only 36 occupational 
keys on the revised blank for men, and 17 occupational keys on 
the blank for women. It permits the counselor then to discuss 
levels of ability, achievement, and aptitude required for a wider 
range of jobs within the interest type, and thus it permits read- 
justments of the student's plans in the light of other pertinent 
data about him. It gives the student a clearer understanding of 
the place of interests in making a vocational choice, because the 
counselor can explicate the student's responses to his earlier 
questions as they relate to an interest type theory, It reduces 
to a minimum any conflict between the student's specific choices 
and the counselor’s alternative suggestions, since both the spe- 
cific choices and the alternative suggestions are assigned to 

reader categories of interest types, where the student can more 
easily see his own status in regard to types of occupations. 

The clinical effectiveness of this alternative plan of interpreta- 
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tion has been demonstrated in the experiences of |!;radnafe \ixv 
dents in supervised clinical training, and in the reaction nf trained 
counselors to the plan. Students are less prone to misinterpret 
the outcomes of the interview; parents can see more clearly the 
relevance of specific educational and vocational Mig|;estirn\s made 
by the counselors; greater flexibility is possible in working oiU 
educational and vocational plans; more satisfaction is e^spressed 
by students with this form of counseling assistance in their voca- 
tional problems. 

No claim of infallibility, however, is made for the plan of 
interest test interpretation. It is not easy to learn, nor will it 
solve certain student problems of inflexible and over^emoiionah 
i2;ed or fixated vocational choices, It requires skillful interview- 
ing and careful explanations* 

There are other aspects in counseling on the basis of mtese^t 
measurement that should be mentioned. The ahsence of a con-' 
sistently significant correlation between specific occupational 
scores and either ability or achievement has already been men- 
tioned. Yet in these studies certain experimental problems remain 
uncontrolled. Clinicians can cite many cases in which a ^tudsmt 
has substantially improved his college grades when he traiinfers 
to a curriculum that trains for an occupation which is within his 
basic and primary interest type. Students trauhfcrrrnj; !unn 
engineering to business administration, from nicdipine to journal’ 
ism, from chemistry to leaching, and succeeding better after sut h 
transfers are familiar to all counselorvS. The giade inerrmeni 
cannot be attributed solely to easier academic competition in the 
second curriculum, since the second curriculum may demand no 
less general academic ability than the first, and may cicnuuul ejif^ 
ferent types of special achievements and aptitudes than the first. 

If the interest measurement can be considered an approxim:itr 
quantification of motivational factors, the following experiment 
would be significant. Choose a group of students having a pr\^ 
mary pattern and a group having a secondary or tertiary pattern 
in the interest type for which a given curriculum offers; specific 
occupational training, Match cases from the two groups on the 
basis of scholastic ability, If the primary pattern in the interest 
type denotes more adequate motivation, the group having xhiu 
pattern should earn better grades than its matched gmup, pro- 
vided no disproportionate factors of disabiliticK or ptohUtm load 
this experimental group. 

Furthermore, when any raw score on an interest key above 
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— .5 sigma can receive the same A grade, there is some question 
of the legitimate use of the Pearsonian correlation in studying 
the relation of occupational interest scores to other and more 
normally distributed variables, such as ability or achievement. 
Correlation ratio or contingency coefficient statistics may be more 
appropriate forms of analysis for these data. It is too early to 
consider occupational interest factors conclusively unrelated to 
curricular factors, in the light of examples to the contrary. 


Conversely some counselors can cite cases in which superior 
or adequate grades are earned in a curriculum that trains for a 
vocation included in an interest group where the student has no 
primary pattern. Yet this need not be too alarming in the light of 
subsequent data about the occupational adjustments of graduates. 
Approximately fifty per cent of all the engineering graduates do 
not continue through life in the technical practice of engineering, 
and the chances are good that many in that fifty per cent have 
primary patterns in interest types other than the technical type. 


This leads to a final clinical apect. The interest type in coum 
seling must be considered in relation to the local institution’s 
curriculum organization in educational guidance. Examples are 
more clearly seen in terms of the blank for women. Many women 
make a primary pattern in the interest type which includes the 
secretarial and office worker keys. The normal curricular path in 
college may be the highly theoretical and technical economics of the 
existing school of commerce or business administration. Yet only 
a small proportion of college girls want to swallow this large dose 
of abstruse economic theory. The primary interest pattern is still 
a true picture of the occupational activities that vi'ould be satis- 
fying prior to marriage; the curriculum may still be excellent 
for professional specialists in business, but the twain probably 
shall not meet happily. General education courses plus a mini- 
mum of training in basic office skills would solve the problem if 
a fiber 1 r" ^ curricular organization. Otherwise 

LsJ CO 1 a ^ ® commercial busi- 

workabuTluIr^ > 


who ,ra familiar with 

the detailed structure of their curricula as well 

iflhpic cv , . , as weu as the curncular 
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THE COURSE IN SELF-APPRAISAL AND CAREERS 
OFFERED TO SENIORS IN THE CHICAGO 
PUBLIC HIGH SCHOOLS’^ 

GRACE MUNSON 

Bureau of Child Study, Chicago Public Schools 

Since February, 1939, seniors in the Chicago high schools have 
been given the opportunity to enroll in a course in Self-Ap- 
praisal and Careers. This course, with its subsequent counseling, 
constitutes the final step in the Adjustment Service. It is the 
culmination of the self-appraisal and educational planning which 
starts early in the elementary grades, is featured in the eighth- 
grade program of articulation between elementary and high 
schools, is an important aspect of the individual counseling at all 
year levels by high-school teacher.s in their daily adjustment 
periods, and is featured again in the third-year program for a re- 
check on mental abilities and reading achievement. These activi- 
ties are given continuity by the cumulative folder system and are 
supplemented all along the way by the individual service and fol- 
low-up studies of both elementary and high-school adjustment 
teachers collaborating with the Bureau of Child Study psycholo- 
gists and demonstrators, for individual cases studies, clinical 
treatment, and consultative service. 

As the Adjustment Service ha.s now been operating in the 
high schools since 1937 and in the elementary schools since 193(5, 
cumulative folders of the fourth-year students of September, 
1940, will contain data assembled over a period of three and a 
half years and in some cases longer. Each year the data will 
extend back farther until ultimately the complete school history 
with many successive measures of mental power and achievement 
will be available for the final guidance step. 

Given in the first half of the fourth year, the course in Self- 
Appraisal and Careers enhances and continues the self-appraisal 
of the earlier years by presenting the concepts of mental growth, 
of individual differences, and of the forces of self determination. 
It makes use of a wide range of scientific measuring techniques 
administered by the adjustment teacher or field psychologists for 

* This article is a summary of the description of the course as pre- 
sented in the Superintendent’s Annual Report for the year 1939 - 40 . 
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the identification of specific areas of mental power, aptitudes, 
academic masteries, and vocational interests. And it teaches the 
techniques for interpreting the profiled results and the various 
conditioning factors. The self-appraisal section of the course 
gives each student a foundation in elementary psychology as a 
background for developing the techniques which will enable him 
to make continued self -appraisal as his pattern of powers and 
achievements changes with new growth and new experiences. 
The new understandings and newly accumulated data together 
with that assembled through the years are now used for making, 
specific immediate plans and tentative future plans for educa- 
tional, vocational and avocational pursuits. 


Career Study Is Dynamic 


The careers section of the course provides studies in specific 
vocational areas using the most recently published books and 
pamphlet series, compilations of current occupational informa- 
tion, regional conferences with selected speakers from lepre- 
sentative vocational areas, personal interviews with these 
speakers, radio broadcasts, and tours. Students acquire knowl- 
edge of the historical development of occupations, their social 
significance, legislative controls and significant trends as a back- 


ground for the development of techniques which will prepare 
them m continue the study of vocations on the basi.s of the new 
experiences and the new skills that may be acquired in the 
changing and diversified world of work, 

The careers section of the course now lacks roots in earlier 
vocational studies comparable with the early development of 
self-appraisal. The problem of adjusting the high-school curric- 
ulum to accommodate such a course for all students earlier than 
the senior year has been studied with great care, since too early 
selection of vocations is detrimental, yet tentative choices should 
govern to some extent educational planning in high school. 
Bepnnings have been made by introducing into selected subject- 
matter courses, study units on the vocational implications of a 

vocations have been added to the 
and Th ? first-year English Reading classes. 

information vocational 

iniormation, the individual counseling at the eighth-grade level 

nd successive high-school levels by adjustment teachers dTvl 

Ziv:n 

some future vocational planning with the students; but 
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the students, having had little opportunity to study careers, are 
unable as yet to contribute intelligently their rightful share of 
the planning. 

The development of a program preliminary to the fourth-year 
course is necessary since an earlier tentative selection of a caicer 
plan will contribute to good mental hygiene by developing secur- 
ity, responsibility, organization of effort, and growth in self 
determination. In this connection, plans arc being formulated to 
drop the third-year testing program to the first half of the second 
year, using the New Chicago Tests of the Primary Mental Abili- 
ties which will yield more diversified data as a basis for self- 
appraisal and counseling; to introduce more specific career 
studies in the second-year curriculum together with a study of 
the total high-school organization of courses and facilities; and 
to establish more clearly defined routines to govern individual 
program-making fiom year to year by division-room counselorb. 

In the senior course the psychological studies and the careers 
studies are presented in somewhat parallel order, one vocational 
area being finally selected for intensive study by each student, 
after the results from the psychological measurements have been 
profiled and interpreted by him. A sample profile is presented in 
Figure I. 

There is little attempt to match a given profile pattern to a 
particular vocation since scientific research has not been able to 
map the specific mental abilities required for insured success in 
a given vocation, and since an attempt to sort individual students 
into vocational pigeon holes would violate the fundamental dem- 
ocratic principles of public education. Yet wise counseling com- 
bines with student freedom of choice based on a knowledge of 
self and of careers, to give each student the security of tentative 
but specific plans. 

The teachers of the course assist students in the formulation 
of such plans through individual counseling as the course pro- 
gresses, using their non-teaching periods for this purpose. In 
the following semester each student goes over his plans again 
with the placement counselor if he seeks employment immedi- 
ately following graduation, or with the senior counselor or the 
adjustment teacher, selecting his college and his first college 
courses, if he plans to continue his schooling. Most adjustment 
offices maintain a library of college information and scholarship 
data. The adjustment teachers arrange for senior visiting days 
at the junior colleges, and confer with their personnel staffs for 


45 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 

Figrure I 

Profile of the teat results for a student in self-appraisal and careers. 

^ Grade-. 


_ Period. 


.. Ti TJ , I ^ « la 10 30 40 W W >0 frO ^ 

tests Date Noims P.K, y 

MENTAL POWER . 


IYA Ail i.’* i 4 ^ ^ w . - ^ n j n 

Primary Mental Abilities 2-40 

1. Perception (P) 

2. Number (N) 35 

3. Verbal (V) 95 

4. Space (S) 99 

5. Memory (M) 75 

6. Induction (1) 98 

7. Deduction (D) 98 

achievement 

Silent Reading (lowa-Adv.) 2-40 

1. Comprehension 99+ 

2. Directed Reading 90 

3. Poetry Comp 99 

4. Word Meaning 99 

5. Sentence Meaning 97 

6. Para. Meaning 91 

7. Loc. of Inf 99 

TOTAL COMP 99 


miiiii 

■IB! 






HS-Content (Iowa) 3-40 

1. English, Lit., Grammar. 

2. Mathematics 

3. Science (Natural) . . . » 

4. History (Soc. Studies) , 


U. ii|s| 


SPECIAL ABILITIES 

Art Judgment (M-S) 4-40. , ,,63 

Music (K-D) 4-40 

Tonal Memory 75 

Quality 97 

Intensity , , . . , 94 

Tonal Movement , . 94 

Time 8 

Rhythm 10 

Pitch . , . ... 96 

Taste 80 

Pitch Imagery 93 

Rhythm Imagery 85 

CLERICAL (NJ.I.P ) 3-40 
L Oral Instructions 40 

2. Classification 80 

3. Arithmetic 70 

4. Copying .80 

5. Checking 90 

6. Piling 97 

7. Problems 98 

TOTAL ,92 
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the orientation program and the transfer of data for such stu- 
dents as plan to attend. They also assist in organizing “College 
Day” when representatives from local and state colleges and 
universities present the advantages of their institutions. 

The course in Self-Appraisal and Careers has been organized 
and serviced by the Bureau of Child Study and the Bureau of 
Occupational Research subject to the advice and guidance of the 
Assistant Superintendent in Charge of High Schools. Confer- 
ences with principals have determined policies for the outlines 
and content of the course while the teachers have contributed 
many valuable devices and suggestions. Each semester the stu- 
dents have made constructive criticism to improve the value of 
the course for the next group. 

Since the course is a five-hour major elective it has not been 
accessible to all seniors following the old program of high-school 
studies. The new program, which will begin to operate in the 
next semester and which allows wider choice of electives, will 
permit more students to enroll. The course shoxild eventually 
be made available to every fourth-year student. 

The following table shows the enrollments in successive 
semesters since the course was established in February, 1939: 


Enrollments in Self-Appraisal and Careers 


Calendar 

No. of Schools 

No. of Classes 

No. of Students 

February, 1939 

32 

74 

2600 

September, 1939 

32 

69 

2500 

February, 1940., . . . 

36 

80 

2800 


The course is taught without a textbook since no high-school 
textbook has been written that covers the psychological studies 
selected for the course and since most textbooks on occupations 
are likely to be out of date by the time they are printed. Instead, 
an extensive bookshelf of reference materials for both teachers and 
pupils is supplied, supplemented by current materials on occupa- 
tions. Students thus have an opportunity to read widely in the 
areas of their interests. To make the books more available to stu- 
dents, several books have been unitized for each school, by divid- 
ing them into from 14 to 43 sections re-mounted in manila covers, 
thus introducing a type of individualized instruction. This year 
a set of 10 reprints on psychological topics, written in popular 
vein by eminent leaders in that field, was supplied in class sets 
to each high school. 

Teachers’ lesson plans and outlines have been worked out and 
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distributed to all of the schools, modified from semester to vsemes- 
ter in accordance with suggestions from teachers and pupils, 
This year, in answer to the demands, student work sheets were 
prepared to accompany the teachers^ outlin^js. They were mimeo- 
graphed and supplied in class sets. 

The Outline of the Course 


The outline of the course, arrived at by successive modifica- 
tions in the light of experience, is given below. It will be revised 
still further as experience indicates the directions in which it 
may be more useful to the students and more adequate in fulfill- 
ing its objectives. Lesson outlines for the teacher have been pre- 
pared for all sections of the course. Special study guides have 
been prepared for the use of students in connection with the 
topics which are starred. Units I and II have been prepared by 
the Bureau of Child Study, Unit III by the Bureau of Occupa- 
tional Research, Unit IV jointly. 


Unit I. Hntroductiqn and Bibliography 

A. Aims and activities of the course 

B. Terminology 
C Bibliography 


Unit 11. Self-Appraisal (To be taught simultaneously with Unit 
suggested that each week, two days be spent on 
Unit II, two on Unit III, and one on testing, Constant inter- 
weaving should be practiced,) 

A, Existence of individual differences 
1. Family history and autobiography 

a. Racial and cultural background 

b. Family traits, vocations and achievements 

c. Health history 

d. Educational history 

e. Hobbies 


^ 2 . 


I. bocial development 

g. Occupational experiences 

h. Plans for the future 

Ph^ical and mental differences betwe 
a, Types of differences 
^ The total personality 
The normal curve 
Applications to the testing progra 
Educational implications 
Vocational implications 


b. 

c. 

d. 

e. 

f. 


people 


connection winf^the*^ topics^ w^hkh^^are'^s^ students in 

teacher have been pre^ SSl T^etS^ot 
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g. Chicago’s plan for the study o£ individual differ- 
ences from the kindergarten through the high 
school 

‘’B. Uses and limitations of standardized tests 
1 How accurate are the test results’ 

2. How useful are the test results for prediction? 

3. Do the tests measure all one’s abilities? 

4. How can the information from them be used most 
effectively? 

5. Can the tests designate the one particular job for which 
each person is exactly fitted? 

6. Study of tests to be given in this course 

a. Description 

b. Why selected 

7. Chicago plan for the study of individual differences 
and for the development of techniques of self-appraisal 
from fourth grade through high school 

C, Psychological factors that must be considered in the inter- 
pretation of test results 
L Maturation and change 

’^a. The process of growing up 

(1) Physical and mental growth 

(2) Laws of natural growth — infancy to maturity 

(3) Influence of the environment on growth 

(a) Effect of frustrations 

(b) Effect of social environment 

(4) Adolescence 

(5) Maturity — the learning ability of adults 
Individual control of the direction of growth 

(1) Habits: our masters or our servants 

(a) Conditioned response 

(b) Deliberate reconditioning 

(2) Development of work habits 

(a) Urge to mastery and completion 

(b) Urge to self-direction 
2. Mastering our environment 

•^a. Human drives and obstacles 

(1) The basic drives 

(2) Psychological bases for the emotions 

(3) Motives derived from basic drives 

(4) The complexity of motives 

(5) Motives as products of the environment, plus 
psychological factors 

(6) The universality of obstacles 

(7) Drives in career planning 

'*^b. Mastery and adjustment — interaction of the indi- 
vidual and the environment 

See footnote on preceding page. 
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(1) Successful adjustment: mastery of one’s prob- 
lems 

(a) Adjustment of environment to self 

(b) Adjustment of self to environment 

(2) Less successful types of adjustment 

(3) Self-appraisal and the choice of adjustment 

(4) Relation to educational and vocational planning 

D. Individual interpretation of the test results 
L Necessary statistical concepts 
'’'a. Percentile rank 
b* Mean 

c. Median 

d. Quartile 

2. Profiles of test results to be made by the student 

a. Construction 

b. Interpretation of test data as samplings 

c. Comparison of abilities and achievements 

3. Aids in interpretation of individual performance on 
each of the following tests: 

Thurstone Primary Mental Abilities 
*b. American Council on Education Psychology Ex- 
amination 

*c. Iowa Silent Reading Test, Advanced 
*d. National Institute of Industrial Psychological Cler- 
ical Examination 

*e. Cleeton Vocational Interest Inventory 

4. Aids in interpretation of the completed profile 

Unit IIL Careers and Occupations (Simultaneously with Unit II) 
A. Man’s interdependence in work 

1. The growth of interdependence 

a. Primitive methods of work 

b. Development of specialization 

c. Effect of specialization 

d. Discussion of our present highly specialized work- 
ing world 

e. Release of human energy for cultural service, and 
leisure-time activities 

2. Evolution and importance of occupational groupings 

a. Development and significance of the merchant guilds 

b. Development of craft guilds 

c. Later history of craft guilds 

d. Present day significance 

(1) Employee organizations 

(2) Employer organizations 

(3) Trade associations 

(4) Professional organizations 

* See footnote on page 48 . 
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3, Socio-economic factors in the study of an occupational 
area 

a. Questionnaire study 

b. Basic attitudes toward occupational rewards other 
than money 

4. Legislation affecting workers 

a. Social Security — old age insurance 

b. Social Security — unemployment compensation 

c. Wage and hours laws 

d. Child labor laws 

B. Significant relations and trends in occupations 

1, Classification of occupations 

a. Importance of study of occupational areas in a broad 
sense as well as of specific occupations 
bf Occupational areas vs. occupational fields 

2. Significance of trends in occupations 

a. Technological 

b. Commercial 

c. Personal and domestic 

d. Professional and semi-professional 

C. Study of an occupation 

1. Relationship between the school subjects and occupa- 
tions related to those subjects 

2. Graphs of life earnings 

3. Case study of an individual 

4. Intensive study of several selected occupations 
(check list or outline for occupational study) 

5. Intensive study of several selected avocations 

D. Techniques in securing and holding work 

1. Channels in finding work 

2, Written application for work 

3, Making an interview 

4. Adjusting to a job 

Unit IV. ^Summary 

A. Development of techniques for self-guidance 

1. Summarizing self-appraisal data 

2. Summarizing data for study of occupations 

B. Schedules for counseling during the ensuing semester 

1. Functions of placement counselor 

2 . Functions of other counselors available in school 

3. Appointments 

C. Application of data to the solution of individual problems 
and the formulation of two plans~one a tentative long- 
range plan, and one a specific plan for immediate actioUi 
both to include provisions for education, vocation, and 
avocation 


See footnote on page 48, 
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D. Evaluation of the course 

Student Work Sheets have been prepared to implement the 
outline of Units I, 11, and IV. The topics are listed below. Over 
2,000 copies of each were distributed to students during the year 
1939-1940. 

1. Introduction and Bibliography, 17 pp, 

2. Physical and Mental Differences, 6 pp. 

3. Standardized Tests, 7 pp. 

4. Process of Growing-Up, 15 pp. 

5. Self-Directed Personality Change, 10 pp. 

6. Human Drives, 14 pp. 

7. Mastery and Adjustment, 10 pp, 

8. Meaning of Percentile Rank 

9. Primary Mental Abilities, 5 pp. 

10. Mental Power as measured by A.CJS. Test, 4 pp. 

11. Reading Ability as measured by the Iowa Silent Read- 
ing Test, Advanced, 5 pp. 

12. Clerical Ability as measured by the N.LI.P. Test, 4 pp. 

13. Vocational Interest, as indicated by the Cleeton Voca- 
tional Interest Inventory, 7 pp. 

14. Summarizing Self-Appraisal Data, 2 pp. 


Battery of Tests for Self-Appraisal Used In 1939*1940 


I. Mental Tests 

A, ^Thurstotie Tests for the Primary Mental Abilities 

1. Perception 

2. Memory 

3. Number 

4. Space 

5. V erbal 

6. Inductive reasoning 

7. Deductive reasoning 

B. ^American Council on Education Psychology Exami- 
nation 

II. Reading Ability 

A, nowa Silent Reading Test, Advanced 
HI, Achievement Tests 

A. *Iowa High School Content Examination 

B. American Council on Education Cooperative General 
Achievement Test 

1. Mathematics 

2. Science 

3. Social Science 

IV. Aptitude Tests, as desired 
A. Clerical 


^National Institute of Industrial Psychology Clerical 
1 est, American Revision 
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B. Mechanical 

1. Detroit Mechanical Aptitudes Examination 

2. Individual Manipulation Tests 

C. Musical 

1, Kwalwasser-Dykema Music Tests 

2, Seashore Measures of Musical Talent 

D. Artistic 

Meier-Seashore Test for Art Judgment 
V. Miscellaneous 

A. Cleeton Vocational Interest Inventory 

B. Business Education Council Personality Rating Sched- 
ule 

C. Kuder Preference Record 

Bookshelves £or Students and Teachers 
The following bookshelves have been set up for students and 
teachers: One set has been furnished to each high school, 
References for Students 

Psychological Books 

William E The Five Shters. New York: W. Morrow and 
Company, 1939. 

Psychological Pamphlets (30 sets to each school) 

■^(Reprints from a series of radio lectures published under the title 
of Psychology Today by the University of Chicago Press, 1932.) 

Garrett, Henry E Psychology Today 

Goodenough, Florence ... , Child Development 

Gesell, Arnold Growth of the Infant Mind 

Watson, John B How to Grow a Personality 

Allport, Floyd H Personality in Our Changing Society 

Cannon, Walter B Flfccts of Strong Emotion 

Warden, Carl J.. . . ... , Animal Drives 

Robinson, Edward S Learning and Forgetting 

Thorndike, Edward L Effects of Rewards and Punishments 

O'Rourke, L. J Matching Men and Occupations 

Occupational Books 

fBrewer, John M. Occupations, Boston: Gmn and Company, 1937. 
tChapman, Paul W. Occupational Guidance, Atlanta: Turner E. 
Smith and Company, 1937, 

tClark, Harold F. Life Earnings, New York: Harper and Brothers, 
1937. 

fFleischman, D. E. An Outline of Careers for Women, Garden City: 
Doubleday, Doran and Company, 1935, 

Giles, I. K. Occupational Civics, New York: Macmillan Company, 
1936, 

’tLyons, George J. and Martin, Harmon C. The Strategy of Job Find- 
ing. New York: Prentice-Hall, Inc,, 1939. 

^ Books added during 1939-1940. 
t Books which have been unitized. 
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National Resouices Committee Tpchtwloj^tc.^! Trends and Nationst 
Policy^ Washington, D, C.: United States Supt, of Docuincnts, 1937. 
Occvpatjoml Outlines on America's Major Occapado/js. Chicago; 

Science Research Associates, 1940, 

tRosengarten, W, Choos/nff Your Li/e Work, Now Yoik: McGraw- 
Hill, Inc., 1936. 

United States Dept, of Commerce Cemus of Busit\e<,s: 193S. Wash- 
ington, D. C.: Bureau of the Census, January. 1937, Also Census 
of Retail Trade, 1936. 

tWilliamson, E. G. Stuc/e/itK and Occupai/OfKV, New Ymk; Henry 
Holt Company, 1937. 

t Ziegler, S. H. and Wildes, Helen J. Choosing an Occupation, Phila- 
delphia; John C. Winston Co., 1937, icvisod edition. 

Occupational Pamphlets 

American Job Series. Chicago: Science Research Associates, 1700 
Prairie Avenue. 19 occupational monographs, 

Are There Opportunities for Women? 1936. 10 pamphlets. Changing 
Patterns in Occupations. 1936. 26 pamphlets. New York: National 
Federation of Business and Professional Women's Clubs* IS19 Broad- 
way. 

Occupational Pamphlets. New York: National Occupational Confer- 
ence, A series of appraisals and abstracts of available literature, 57 
pamphlets. 

Occupational Research ReportSk Chicago; National Youth Adminis- 
tration of Illinois, Merchandise Mait 29 pamphlets. 

Occupational Briefs, Briefs compiled by the National Youth Adminis- 
tration on the occupations included in the reports above. 

Guidance Leaffets. Washington, D, C.: United Slates Printing Ofiice, 
1936 19 pamphlets. 

Success — Vocational information seiles. Directed by Chloris Shade, 
Joliet Township High School. Chicago; Morgan-Dillon and Company, 
55 pamphlets. 

Bibliographical Belps 

Bennett, Wilma. Occupationa/ and Vocational Guidance — A Source 
List of Pamphlet Material, New York; H. W, Wilson Company, 1936, 
revised edition. 

'•^Massachusetts Youth Administration, Bibliography of Occupational 
and Apprenticeship Information. Boston; 31 St. James Avenue, 1937. 
101 pp Comprehensive list of magazine articles. 

Paiker, Willard B Books About Jobs, Published for the National 
Occupational Conference by the American Library Association, Chi- 
cago, 1936. 

Price, Willodeen and Ticen, Zelma E, Index to Vocations. New York; 
H. W. Wilson Company, 1936, revised edition. 

Sibhogiaphy of References on Vocational Guidance /or Girls and 
Women. United States Office of Education, Washington; Vocational 
Division, 1936, revised, 13 pp. Lists bibliographies, studies and in- 
vestigations. 

* Books added during 19394940, 
t Books which have been unitized. 
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Vocational Guides Chicago: Science Research Associates. A monthly 
bibliography of occupational books and ai tides. 

Research Services 

Occupational Card File on Current Local Data Chicago; Bureau of 
Occupational Research, Board of Education. 

Cumulative Bulletin Series. Chicago: Bureau of Occupational Re- 
search, Board of Education. 

Special Research Reports^ Chicago; Placement Clearance Center, a 
division of the Bureau of Occupational Research, Board of Education. 

References for Teachers 

Psychological Books 

Bingham, Walter V. Aptitudes and Aptitude Testing. New York; 
Harper and Brothers, 1937. 

Paterson, Donald O., Schneidler, Gwendolen G>, and Williamson, E, G. 
Student Guidance Techniques. New York; McGraw-Hill, Inc. 1938. 
Shaffer, Lawrence F. The Psychology of Adjustineni Boston: 
Houghton, Mifflin Company, 1936. 

Strang, Ruth M. Role of Teacher in Personnel Work. New York: 
Teachers College, Columbia University, 1936. 

Occupational Books 

Lincoln, Mildied E. A Short List of References on Methods of 
Teaching Occupations. New York: National Occupational Confer- 
ence. Mimeographed, 3 pp. Free upon request. 

Lincoln, Mildred E. and Brewer, John M. How to Teach Occupa- 
tions. Boston: Ginn and Company, 1937. 

The Reactions of Students 

It is too early to obtain an adequate evaluation of the course. 
If the opinion of the students is a criterion (and who doubts that 
it is an important component?) the course is highly successful. 
The reactions of students indicate their deep sense of responsi- 
bility at this level of high school training, the changes in their 
viewpoints engendered by the course, and their gratitude both for 
the new knowledge acquired, and for the personal guidance from 
the fine men and women who have taught the course. A few of 
the student comments are presented below; 

“I feel I have benefited by almost every topic and dis- 
cussion in this course in Careers, but several parts have been 
very helpful. I enjoyed all the tests, and the experience of 
having had them helped me when I applied for a position at 
the Continental Bank. Four tests were required and one was 
almost identical to those we have been taking. All the tests 
at the bank were somewhat like those we have been taking 
and I was much more confident than I would have beenj if 
the work had been new to me. Another part of the course 
that I feel has been of great help to me has been hearing 
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about various occupations. It is much easier to decide upon 
a vocation for yourself^ one that you think you will like, after 
you hear the good and bad points about many vocations. I 
have a much clearer idea of what I would like to be in the 
future than I had before I took Careers.'* 

Gloria K., Ilirsch High School 

“The survey of different vocations and professions has 
been most helpful to me. It never occurred to me that one 
vocation could branch into so many specific fields. The 
Careers class threw light on many subjects concerning which 
I was in the dark, such as the present and future demands in 
the labor market, the demand of the employer upon his em- 
ployee, and the amount of education needed to get along in 
a vocation,” 

. Frances D,, Harper High School 

“This course of Self-Appraisal which you are offering is 
very good in building character, citimis, and regular ladies 
and gentlemen. Now I don*t say this just to be your good 
friend because it is everything that I mentioned above and 
more. 

“One of the things that struck me most was the way you 
treat the pupils. Because there is nothing like having a regu- 
lar guy talking to another regular guy. 

“After three and a half years of bumming, cutting, etc., 
this course brought me to my senses. I don't know what it 
was — whether it was the tests, or the homely philosophy — 
but the course was interesting. 

“And now in ending I want to thank you for making this 
change in me. And later in life I’ll come up and give you a 
visit, Maybe ITl be a bum on Madison Street or a big shot 
on Michigan Boulevard, I’ll always come and visit the regu- 
lar guy and at the same time ask him for advice.*’ 

Ted T„ Harrison High School 

“Very few of those who finish high school ever sit down 
and take an inventory of themselves. The talks students 
have with the counseling teacher make you gather your wits 
about you and make you think of how to approach your em- 
ployer. Those who are backward and bashful come out of 
their shell, due mostly to the reassurance of the teacher who 
gives them a boost upward,” 

Margaret I„ Marshall High School 

The most important part of the Careers course is the 
making of the Career book on the selected occupations, be- 
cause it helps you find out all about the occupation and to 
make sure you will fit in that Hue of work. After making my 
book on careers m pattern making, I found I needed to brush 
up on a few technical things. Some students found out that 
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they never will or could fit in the occupation they had first 
chosen, and they have had time to make a better choice.” 

Chester L., Steinmetz High School 

”Of all the courses I have had, one of the most important 
subjects for me and for the development of my character, has 
been “Self-Appraisal and Careers,” It is a subject which we 
have to give a lot of thought to, with quite a bit of brain 
work. It has been helpful in many ways: (1) it makes one 
think fully on the future when he or she leaves education and 
goes into the open world; (2) it helps one to know people 
and to understand them much better; (3) it helps one dis- 
cover the vocation into which he will best fit; (4) it helps one 
get a clearer view of the world and of occupations,” 

Kathleen M,, Waller High School 

Handbooks and Bulletins 

More complete information concerning the course in Self- 
Appraisal and Careers will be found in mimeographed bulletins 
available from the Board of Education in the City of Chicago. The 
bulletins may be obtained for the cost of mailing (75c) by writing 
to the Bureau of Child Study, 228 N. La Salle St, 

Prepared jointly by the Bureau of Child Study and the Bureau 
of Occupational Research: 

Handbook on Self-Appraisal and Careers, 17 pp. 

Teachers* Outlines for Self-Appraisal and Careers, 86 Pt'- 

Prepared by the Bureau of Child Study; 

Student Work Sheets for Self-Appraisal and Carceis, lOS pp. 
Handbook of Norms, 30 pp. 

Handbook on Scoring Procedures, 36 pp. 

High-School Teacher’s Devices and Suggestions (Subject; Self- 
Appraisal and Careers, A Bulletin issued by the Superintendent 'of 
Schools), 19 pp. 

Bureau of Child Study Annual Report, Part V, High-School Self- 
Appraisal and Careers Course, June, 1939, 9 pp. 

Service Bulletin No. 1, 1940, Methods of Presenting the Course in 
Self-Appraisal and Caieers, 

Prepared by the Bureau of Occupational Research : 

Cumulative Bulletin Series; 

Series I Educational Facilities, 30 pp. 

Series II Occupational Information, 23 pp. 

Series III Significant Trends, 3 pp. 

Series IV Pertinent Legislation, 11 pp. 

S7 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 

references 

Johnson, William H , Annual Report of the Superintendent of Schooh, 
Chicago: 1937-38 P, 339, 

, Annus/ Report of the Superintendent of Schools. Chicago: 

1938- 39. Pp. 176-79 

. Annua/ Report of the Superintendent of Schools. Chicago: 

1939- 40. 

. ^'Adjustment Service In the Chicago High Schools/* Educa- 

tional Adn3/ny5tra^/(?n and Supervision, XXIII (Oct 1937), 513-20, 

. "Adjustment Service in High Schools," A menca ri Schoo/ 

Board Journah XCVI (May, 1938), 30-32. 

. "Adjustment Teacher Service in Chicago Elementary Schools/’ 

The Elementary School Journal, XXXVIII (Dec, 1937) 264-7t 

. "Guidance, Counseling and Adjustment," The School Execu^ 

the, LVI (May, 1937), 333-34: 

. "New Self-Appraisal and Career Study Course in the Chi- 
cago High Schools/’ School and Society, XLIX (May 20, 1939), 627-31. 

. "Place of Guidance, Counseling and Adjustment in the Sec- 
ondary Schools, ''The North Central Association Quarterly, XII (Jan., 
1938), 369-72, 

Munson, Grace, "Adjustment Service of Chicago High Schools," Occu- 
pations, XVII (Feb., 1939), 389-94. 

. "Adjustment Service — Chicago Schools," Educational Method, 

XIX (March, 1940), 327-35. 

Schloerh, Lester J., "N.Y.A. Occupational Monographs," Chicago 
School Journal, XXI (Jan.-Feb., 1940), 180-8L 

. "Placement Counselors in Chicago Schools,” Occupations, 

XVIII (Feb., 1940), 387. 


58 



primary mental abilities and aviation 

MAINTENANCE COURSES* 

WILLARD HARRELL, University ol lUinois 
and 

RICHARD FAUBION, Au Corps Technical Schools 

This investigation is the third of a series designed to deter- 
mine the optimal pattern of abilities for mechanical work, The 
first study, "A Factor Analysis of Mechanical Ability Tests" (I) 
suggested that the principal component of the Minnesota secies of 
mechanical tests is the Space factor. A .second factor, tentatively 
identified as the Perceptual, was present in that battery. A Man- 
ual Agility factor was also isolated. None of the Minnesota teats 
possessed a significant weight for this Agility factor. The most 
practical conclusion from thi.s first study was that certain paper 
and pencil tests will measure equally validly each of the factors 
present in more cumbersomoly-administcred mechanical tests. 
The second study, "Selection Tests for Aviation Mechan- 
ics (2),” consequently involved only paper and pencil tests. This 
second study was started after the publication of Thurstone's 
monograph, "Primary Mental Abilities (3)," but was begun before 
his Experimental Battery of Primary Mental Ability Tests (4) 
became available. Nine of the tests from the monograph supple- 
ment were included along with 29 other sub-tests. These were 
taken by 84 basic instruction students of the Air Corps Technical 
Schools, Basic instruction grades from each of five aviation 
maintenance courses with a total duration of eight weeks formed 
external criteria. These course grades were the criteria for both 
the second study and for the third, the subject of this paper- 
Air Corps Technical School students take these five basic 
instruction courses regardless of later specialization in radio, 
photography, airplane mechanics, parachute rigging or other 
advanced specialties. The five basic courses are Shop Mathemat- 
ics, Mechanical Drafting and Blueprint Reading, Air Corps Fun- 
damentals, Elements of Metalwork, and Elements of Electricity. 
The names are perhaps sufficiently definitive except for two of 

*This report is of a study sponsored jointly by the Trade Teat Depart- 
ment, Air Corps Technical Schools, and the University of Illinois’ Gradu- 
ate Research Committee. The paper was read at the Mid-Western 
Psychological Association, May 4, 1940, at the University of Chicago, 
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the courses. Air Corps Fundamentals is unlike the others in that 
it does not entail mechanical problems. It is made up of the 
study of Air Corps rules and nomenclature. Shop Mathematics 
includes the following topics: addition, subtraction, niultiplica^ 
tion, and division; fractions and decimals; denominate numbers 
and mensuration; formulas and tables; shop trigonometry; ap- 
plied problems. 

Entrance to the Air Corps Technical Schools is restricted to 
soldiers in the United States Army. Consequently the minimum 
age IS 18 years. The education requirement is graduation from 
high school or the equivalent. A minimum Army Alpha peicentile 
rank of 75 is required. Percentiles here are based on the Army 
population. The percentile rank of 75 corresponds to an Otis 
I.Q. of 100. 

One hundred and five soldiers, students of the Air Corps 
Technical Schools, were given Thurstone’s Experimental Battery 
of Primary Mental Ability Tests (4), and two additional tests 
found predictive in the previous study (2) — Surface Develop- 
ment and Punched Holes. Army Alpha scores were also available 
since they are used as an entrance requirement. About half the 
group was in the advanced phase of Airplane Mechanics, and the 
other was in Radio Mechanics. The age range was 18-39 with a 
mode of 19. The range for years of formal schooling was 9-15. 
Sixty-four had completed high school but had gone no further. 

The classification of students into the various advanced phases, 
as well as their selection, might be considered a test problem in 
part, but only the selection angle will be considered here. Re- 
sults are becoming available from tests given to 600 students to 
provide sufficiently large samples to trace the correlation between 
tests and several advanced phases. 

It is recognized that the course grades are not perfect criteria. 
They are complex, but since they consistently correlate signifi- 
cantly with several tests, they probably possess a reasonable 
amount of validity, One objective criterion — a machine shop prod- 
uct ^has been developed which is hoped to have a satisfactory 
reliability. Other practical criteria and objective information 
criteria are planned. 

The reliability has been estimated by the split-half method for 
each of the sub-tests correlating as high as .30 with a criterion. 
These coefficients are shown in Table HI. 

Only two of the seven Alpha sub-tests with significatrt correla- 
tions with any grade, namely Addition and Analogies, have a 
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reliability above ,90. Alpha Arithmetic with reliability of *60 
is lowest. 

Three of the Primary Mental Ability tests, Completion, Arith- 
metic, and Number Series have reliabilities of less than .90 but 
more than .80. From an item analysis, showing the correlation of 
each item in the PMA battery with total sub-test score, the 
relatively low reliability in these three cases is probably due 
in part to the items not being arranged in order of their difficulty. 

Four of the PMA tests, Addition, Same-Opposites, Cards, and 
Figures, have reliabilities above .97. These high reliabilities may 
be partially explained by the items within each of the tests being 
practically of equal difficulty. 

Comparison of A. C. T. S. Students with High School Seniors 

A comparison has been made between the PMA scores for Air 
Corps Technical School students and the norms published for 300 
Hyde Park High School (Chicago) seniors. Table IV shows this 
comparison. Critical ratios have been calculated from the differ- 
ences between means. CR*s for Number and Memory are less than 
.30. Hyde Park seniors have higher Perceptual, Verbal, and 
Induction scores. Air Corps Technical School students have 
higher Reasoning and Space scores. 

It is difficult to interpret these results because it is not pos- 
sible to say exactly what selective agents are at work in the Air 
Corps Technical School. The most obvious ones are, being a 
soldier, choice by a commandant which presumably means interest 
in mechanical work, completion of high school, and having an 
Alpha Army percentile rank of 75, 

A difficulty with the Reasoning or D score is that one of the 
tests, Mechanical Movements, on which the D score depends, also 
possesses a significant weight in another factor which from an 
unpublished factor analysis by the writers seems to be Knowl- 
edge of Mechanical Processes. Since the present group is 
selected in part on their interest and, presumably, knowledge of 
mechanical processes, this would increase the Mechanical Move- 
ments score and consequently the D score, without demonstrat- 
ing that they are better reasoners than the Hyde Park seniors. 

Results with Primary Mental Abilities Tests 

All of the PMA scores were obtained from adding test scores. 
Five of the seven, all but Perception and Memory, correlate sig- 
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nificantly with at least one of four basic instruction grades 
Table I lists the product-moment correlation coefficients A 
Bignificant correlation is considered to bo one wheie the coeffi- 
cient js at least four times its probable ciror For 105 cases this 
Is a correlation of 24 

Elements of Metalwork does not correlate significantly with 
any of the tests, but it did in the second study referred to above. 
The test correlations with Shop Mathematics and with Mechan- 
ical Drafting appear quite similar to those in the previous study. 
In both groups, Addition, Number Series, and Surface Devel- 
opment correlated significantly with Shop Mathematics; and In 
each study Mechanical Movements, Suifacc Development, and 
Punched Holes with Mechanical Drafting. There are stronger 
correlations with Electricity, and with Air Corps Fundamentals 
in this study than in the second A possible explanation is that 
the present battery has more Verbal tests, and these correlate 
significantly with each of those two courses. More important is 
that the greater dispersion for age and schooling of the present 
group tends to increase the correlations. 

Looking again at Table I, the Number factor correlates sig- 
nificantly only with Shop Mathematics; Space corielates signifi- 
cantly with Shop Mathematics, and with Mechanical Drafting; 
Induction with Shop Mathematics, Electricity, and Mechanical 
Drafting; while Reasoning and the Verbal score correlate sig- 
nificantly with each of the four basic grades 

Mutiple correlation coefficients using only significant zero- 
order coefficients have been computed between PMA scores and 
each of four basic grades These may be compared with correla- 
tions of Alpha total with the four criteria grades. The multiple 
R's from the factor scores are 46 with Shop Mathematics, 57 
with Electricity, .60 with Mechanical Drafting; and ,36 with Air 
Corps Fundamentals Corresponding values for Alpha total are 
.31, 47, .30, and .41 These multiple R’s, as well as others to be 
mentioned later, would be expected to be leas in other samples by 
the shrinkage effect if the same regression formulas were used 
The multiple correlation between four factor scores, Verbal, 
Space, Induction, and Reasoning, is 63 with a composite basic 
grade obtained from adding grades in Shop Mathematics, Elec- 
tricity, and Mechanical Drafting, Alpha total correlates ,45 with 
this same composite The zero order correlations with this com- 
posite grade are given m Table VI. Table V shows the inter- 
correlations of five factor scores. 
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Results are shown in Table II for those sub-tests which cor- 
relate 30 or more with one of the basic grades This is five times 
the probable error 


Conclusions 

The Air Corps Technical Schools are planning to supplement 
their test selection in line with these reaiilts; and they also expect 
to establish test standards for classification from future sludlea, 

Wq have come to the conclusion from this and other studies 
that there is no one separate factor for a mechanical ability 
Rather, there are several factors which are more or less proiH' 
Inent in mechanical work, their pattern depending on its type 
and complexity and on the point reached in the learning curve 

A Perceptual factor, although piesent in several so-called 
Mechanical Aptitude tests, is probably related to mechanical 
work, borrowing an expression from Holzinger, as an Arti-factor 
The Verbal factor has been shown to be evident in training for 
mechanical work of relatively great complexity Among the more 
important factors in mechanical operations arc Space, one or 
two Reasoning factors, and Knowledge of Mechanical Processes 
A Manual Agility factor is present in routine jobs where indi- 
vidual difEercnces depend on the manipulation of objects such 
as nuts and bolts 


TABLE I 

Product-Moment Correlation Coefficients Between 5 'Trimary Mental 
Ability” Scores and 4 Aviation Maintenance Courses for 105 Soldiers* 



Shop 

Math 

Elec- 

tricity 

Blue Print 
Reading and 
Mech Drattg 

Air Corps 
Funda 
mentals 

14 N (Addition, Multiplication) 

37 

17 

00 

U 

17 V (Completion, Same Opposites) 

28 

SI 

37 

33 

20 S (Cards, Figures) 

27 I (Letter Grouping, Marks, 

25 

17 

36 

02 

Number Patterns) 

31 D (Arithmetic, Number Scnce, 

33 

20 

41 

20 

Mechanical Movementa) 

PEt — ,05 wheie r = .50 

PEp 06 where r = ,20 

26 

40 

1 

54 

24 


* Decimal points have been omitted before each coeifieJent, 
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TABLE U 

Product-Moment Corrcyation Coelfiticnta Between 19 Teats atul 4 
Aviation Maintenance Coursca for 105 Soldicis*'^ 



Shop 

Math 

Kicc 

tnciVy 

Blue Print 
Reading amt 
Mcch DrariR 

Air Corps 
Fundn 
nicntals 

1 Alpha Addition 

33 

06 

- 10 

-04 

2 Alpha Arithmetic 

21 

35 

30 

27 

3 Alpha Common Sense 

13 

31 

06 

06 

4 Alpha Word Opposites 

13 

43 

24 

38 

S Alpha Mixed Sentences 

17 

43 ' 

23 

34 

6 Alpha Number Senes 

37 

IS 

24 

1 07 

7 Alpha Atialagies 

26 

35 

23 

1 39 

12 Thuvstone Addition 

39 

23 

08 

24 

15 Thnrstone Coinpletlon 

23 

47 

39 

21 

16 Thurslone Same Opposites 

27 

45 

30 1 

34 

IS. Thuratonc Cards 

23 

la 

32 ' 

00 

19. Thurstone Figures 

21 

12 

33 

05 

24 Thurstone Letter Grouping 

23 

25 

31 

22 

26 Thurstone Number Patterns 

30 

18 

32 

05 

28 Thurstone Arithmetic 

31 

49 

49 

29 

29 Thurstone Number Senes 

22 

33 

39 

17 

30 Thurstone Mechanical Movements 08 

12 

40 

10 

32. Thurstone Punched Holes 

15 

16 

41 

06 

33 Thurstone Surface Development 

35 

21 

50 

02 


^ Decimal points have been, omitted before each coefficient, 


TABLE HI 


Test Reliabilities by the SpUt-Hall Method (Stepped-up) 




N = 

103 


1 

Alpha Addition 

96 

19, Thurstone Figures 

” 99 

2 

Alpha Arithmetic 

60 

24 Thuistone Letter Grouping .91 

3 

Alpha Common Sense 

87 

26 Thujstone Number 


4. 

Alpha Word Opposites, 

88 

PatteriiB 

92 

5 

Alpha Mined Sentences 

BO 

2B. Thurstone Authmetlc 

.87 

6 

Alpha Number Senes 

84 

29, Thurstone Number Series, 

.87 

7 

Alpha Analogies 

93 

1 30 Thuistone Mechanical 


12 

Thurstone Addition 

98 

Movements 

93 

15 

Thurstone Completion , 

.80 

32. Thurstone Punched Holes 

69 

16. 

Thurstone Same-Oppositcg 

99 

33 Thurstone Surface 


18 

Thurstone Cards , 

90 

Development , , 

.95 
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TABLE IV 

Comparison Between 300 Hyde Park Iligli School Seniors and 105 Air 
Corps Technical School Students lu “Primary Mental Abilities" 


Hyde Park ACTS 

Seniors Students CR 

Mean S D Mean S D 

~152 23 75 137 09 16 sf 7 0-1 

119 5 30 00 118 78 27 80 0 22 

84 5 27 50 75 16 19 95 3 72 

109 5 35.00 125 23 34 14 4 04H- 

15 5 7 25 15 65 7 67 0 18-h 

35 5 9 00 28 46 8 76 7 04 

_ 54 5 18^ 68 85 18 48 6 69 h 

TABLE V 

Prodnct-Moinciit Coiielation Coefficients 
Among Pactoi Scores foi 105 Soldieis 

- ~N y- ~ f ~ 

‘V 31 

S 30 15 

I 33 28 39 

D 20 33 41 54 

TABLE VI 

Product-Moment Correlation Coefficients Between Tests and a Composite 
Basic Grade Composed of Shop Math, Elccliicity, anti 
Mechanical Drafting 




N = 

105 



1 

Alpha Addition 

11 

'19 

Thunlone Figures 

.29 

2 

Alpha Arithmetic 

.34 

20 

Sp.ico 

34 

3 

Alpha Common Sense 

17 

24 

Thuratone Letter Grouping ,34 

4 

Alpha Opposites 

.31 

26 

Thurstonc Number 


5 

Alpha Mixed Sentences 

32 


Patterns 

.36 

6 

Alpha Number Senes 

35 

27 

Induction 

44 

7 

Alpha Aualoeles 

.35 

28 

Thuratone Arithmetic 

.53 


Alpha Total 

45 

29 

Thurstonc Number Series 

.39 

12 

Thill stone Addition 

.29 

30 

Tluirstonc Mechanical 


14, Number 

23 


Movements , 

26 

15 

Thuratone Completion 

.44 

31 

Deduction , 

50 

16 

Thuratone Same-Opposites 

41 

32 

Thuratone Punched Hoiea, 

.31 

17 

Verbal 

47 

33 

Thuratone Surface 


18 

Thuratone Cards 

32 


Development 

47 


Perception 

Number 

Verbal 

Space 

Memory 

Induction 

Reaaonme 
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A COMPARISON OF THE ORIGINAL AND REVISED 
STANFORD BINET INTELLIGENCE SCALES 

MARTIN L REYMERT AND RALPH K MEISTER 
TAe Moosciicajt Lafioratoiy lor C/iifd Re&earcft 

The present sLudy is an attempt to compare the original and 
the revised Stanford-Binct Intelligence Scales The data have 
been obtained from 440 Mooseheart children, each of whom has 
had from two to nine examinations The population of tests 
comprises 958 administrations of the oiiginal scale and 823 
administrations of the revised, The testing was done by trained 
clinical psychologists of the Laboratory staff The children are 
all normal and have been drawn Evom eveiy state in the Union, 
predominantly fiom the Middle West 

The following items weie iccordcd, The child’s name, birth- 
date, the dale of administration of the test, the I Q laliiig 
obtained, the MA, the C.A , the basal yeai scoie, the highest 
level of success and the amount of scattci The time intcival be- 
tween admiiiistiationb and the diicction and amount of deviation 
from the first I Q lating to the second wcic obtained for each 
pair of successive administrations 

To compare the equivalence of ratings fiom one scale to the 
other with the respective reliabilities of the scales, using the 
same population, two groups of clnldien were chosen All had 
had at least two examinations with the original scale and two 
with the revised. However, Group A of Table I had talcen the 
L foim of the revised scales fiist while Gioup B had taken the 
M form first, 


TABLE I 

Correlations Between the Vaiious Forma of the Stanford-Bmet Scales 

for Constant Populations 
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The reliability coeRicients for the original scale in both 
groups are above .80. So are the corielalionfl between the two 
forms of the revised scales which have been considered here as 
analogous to reliability coefficients. The conelationa between 
the original scale and each form of the revised, which give an 
estimate of the equivalence of ratings, are not signiricnnUy dif- 
ferent‘ although the correlation with the M form is lower. Thus 
it can be said that the reliabilities for both scales and the correla- 
tions between both scales are essentially high and equal 

Table II, which shows the correlations between the original 
scale and the two forma of the revised, and their respective reli- 
abilities, using all the population that was available with no 
attempt to keep the composition of the groups the same, gives 
estimates that are all high Tvith but one exception 


TABLE 11 

Correlations Between the Various Forms ol the Slanfoid-Dinel Scalci 
for Populations of Variable Compooilion 


Scales Correlated 
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Av Age at First Teat (in years) 

10 3 

97 
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12,1 
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10.3 
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13 

1.9 

19 

1 28 

1 1 7 

1 2 

1.0 


* An admlmstration of an alternate form was included between the 
two forms correlated Therefore these correlations arc not between suc- 
cessive administrations as arc the others 


The estimate of correlation between the M form and the L 
form of the revised scale is significantly lower than any of the 
other estimates However, since an estimate of this same correla- 

^No PEt is given m this study since an improved tcchnltiue (Ridci, 
to pp 64'S5 ) has been used to determine whether tlie dlfrorciicc between 
correlation coefficients la significant. The use of the P.E„ is n crudely 
approximative method at best and in this particular case it is erroneous 
since the aaaumpUorv of a normal distribution of correlation coefficients 
LB probably violated in this case 
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tion obtained m Group B (Table I) is high, ,89, the lower coeffi- 
cient heie, may be due to the particulai sample taken, 

Another view of the equivalence of ratings from scale to 
scale can be obtained from a study of the deviations fiom admin- 
istration to administration, The lesults presented in Tc'ible II, 
where the grouping is accoiding to I Q classification, show that 
within each scale and between scales the individuals with the 
lowest I Q 'a gam most upon retest, those of average I Q gam 
some, while those of highest I.Q actually lose. 


TABLE HI 

Deviations Between Ratings m Successive Admmistiations Accoiding 

to I Q, ClassiJicationa 
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We have here the expected tendency for the extremes of the 
distribution to migrate toward the mean with euccesBlve reteat- 
inga (regression). 

In Table IV where the deviations are classified according to 
the length of interval between tests, the mean of the absolute 
deviations increases as the interval becomes longer in every case 
but one 

In that case, this reversal of tendency may be discounted in 
view of the small number of cases (10). It is concluded that the 
longer the interval between successive administrations, the 
greater the discrepancies in the ratings 

Table V, which gives for both scales the relation between 
the size of deviation and the number of tests taken, shows that 
the mean of the absolute deviations decreases slightly with buc-* 
ceasive testa for the revised scale and does the same for the 
original scale with one exception 
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TABLE XV 

Dcvialjone in Ralinga According to the Length of Interval Between 

Adiiiini<ttralions 
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TABLE V 

Deviations in Ratings Between Successive Admiiiisiratioiis in Relation 

to the Nnmbei ol TcbIb Taken 
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In general, with continued retesting, the discrepanclcB between 
successive admlmetratioivB tend to become slightly smaller, 

Table VI, which presents the deviationa according to the 
chronological age of the individual at the time of the first test, 
shows a different trend of deviations with age for each scale 
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TABLE VI 

Deviations m Ratings According to the Age of the Individual 

at the First Test 
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With the original scale, the mean of the absolute deviations 
IS a maximum m the middle age group; with the revUed acalej it 
decreases with age; and, between the original and revised scales, 
it increases with age In considering the net gain upon retesting, 
it is found that for the ouginal scale there is a small increase in 
net gam with age^ Foi the revised scale there is a decrease in 
the amount of gain, and between the oiigmal and the revised 
there is a aubslantial increase in net gam with age, In no instance 
is there a net loss 


Changes in Dispersion 

In studying changes in diapeision of the I Q distributions 
from test to retest in order to estimate how well the test will 
discriminate between members of a group upon retest, it was 
thought desirable to keep the population m any paitlciilar com- 
parison constant to avoid any change in dispersion due to a 
change in the composition of the gioup Four groups were used 

Groups A and C show the changes m dispersion on the orig- 
inal scale with one and two leUsts respectively; Groups B and D 
do the same for the revised scale. With successive administrations 
of the revised scales, the standard deviations decreased and in 
Group D this decrease was significant. This is what might be 
expected since there should be a regression toward the mean 
upon retesting To the extent that there is regression a given test 
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table VU 


Changes m the Dispersions of I.Q DiBliibuUona witli Retesting 
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discriiumates less well between the members of a group upon 
retest 

However, in the original scale, the sUndaid deviation on the 
first test distribution is smaller and significantly smaller than 
those of either the second or third admiiiistrations This is true 
for both Groups B and D These latter results may seem con- 
trary to expectation, but it should be icmembered that these 
retests occurred a year later on the average and thus the child 
was a year older, It la known that there is an increase in vai labil- 
ity with mean test performance In othei words, as children 
grow older they tend to be more variable. The opeiation of this 
factor tended to mask the predilection for the disti ibiilion to 
regress toward the mean in the original scale, while m the revised 
scale the regression toward the mean was sufficiently great to 
obscure the opposite tendency, From a practical standpoint, then, 
it appears that with a given group, the discriminal ability of the 
original scale increases slightly upon. leteat while that of the 
revised scale decreases. 

In. the Investigation of scatter, this term will be defined as the 
number of age levels through which an individual had to be 
tested to obtain his rating, from the level at which he passed all 
tests to and including the one at which he failed alh His scatter 
as defined above is larger than his range of successea by one age 
level Scatter is used here as an approximate measure of the time 
taken to administer the test,® In general, the more levels over 

which an individual scatters, the longer it takes to administer the 
test 

The amount of scatter Is limited by the number of age levels 


A direct measuie of the time required, such as the tiae of a 
atop-watcK could not be employed since this study was obtained from 
records which did not contain such mFormation 


72 



ORIGINAL AND REVISED STANFORD BINET SCALES 


present in the test and by the age level at which the subject 
obtains liis basal yeai score, i e , an individual getting a basal 
score at Year XII on the original scale cannot scatter more than 
four test groups since there are no more For this reason, the 
amount of scatter for both scales has been analyzed according 
to the basal year scores obtained This airaiigement makes 
explicit any limitation of scattei by the ceiling of the test 


TABLE VIII 

Amount of Scatter ClassiAcd Accoichng to the Basal Year 

Scoica Obtained 
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Table VIII shows that the scattei increases to a maximum in 
the middle range. The maximum scatter is nt basal year VII for 
the original scale and basal yeai IX for the revised scale It is at 
these points that the ceiling of the test begins to limit the amount 
of scatter Since theie are fewer test groups at the highci age 
levels in the original scale, it might be expected that this ceiling 
would make its influence felt earlier, This is the case. The re- 
vised scale has the grealei scatter throughout, piobably as a 
result of the increased number of tests in it At basal age seven, 
this difference in scatter which has been only slight increases 
somewhat 

According to these results it would seem that the revised scale 
in general takes a longer time to administer This is in agreement 
with the results reported by Knigman (6) No evaluation can be 
made of this finding, since it is not known to what extent the 
longer testing time lesulls in increased accuracy of the rating 
obtained. 


Inversions in Bass! Year Scores 
Inversions in basal year scores, le, instances in which an 
individual on a later test makes a lowei basal year score than on 
his first, were studied since they cast some doubt upon the assump- 
tion that an individual would answer correctly all those items 
below his basal year level In the original scales such inverstons 
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occur m four per cent of tlie total possible instances (24 times 
out of 589) In the revised scale, they occur in nine per cent of 
the total possible instances (35 times out of 412) This difference 
18 not statistically significant 

/nversioris in Item Level 

Inversions In item level or skips, i e , instances in which the 
individual failed all the tests at one age level but succeeded on 
one or more tests on a higher level weie noted The assumption 
m testing is that the individual will not succeed in any test 
beyond the level at which he fails all This assumption and the 
pressing demand of time economy militate against testing the 
child beyond the level at which he fails all the testa so that these 
skips are not so frequent as they might otherwise be To the 
extent that the child ta not given opportunity to perfoim on tests 
at a higher level where he sometimes achieves a inndom success, 
his rating vs an undei estimation of his true ability 

In the original scale such skips occurred in font pei cent of 
the tests (40 times out of a possible 958) i In the revised scales 
they occurred a little less than one pei cent of the time (eight 
times out of a possible 823) The diffeience hefcv/cen these propor- 
tions is significant, Apparently the grouping of tests, fiom the 
viewpoint of avoiding such skips, has been much better in the 
revised scales 

Validity of Mental Year Groupings 
To test whether the grouping of test items by mental years if? 
such as to represent for each year's grouping the normal per- 
formance of children of that chronological age, the basal year 
scores were analyzed according to the average age of children 
achieving those scores Table IX, giving the mean chionological 


TABLE IX 

Average Ages of Children Making Varioua Basal Year Scores 
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ages of children malting the various basal year acoies, allows that 
the mean ages ace in every case significantly higher than tlie ycai 
level indicated by the score 

Within the same basal year group, the mean age for the 
original scale is in no case significantly diffeient from the mean 
age for the revised scale, indicating that though both scales do 
not meet the foiegoiiig criterion for the grouping of test items, 
one is no better than the other 

Summary 

The data weie gathered from 440 normal children who had 
taken a total of 958 original and 823 revised Stanfoid'Binot exani' 
illations The results indicate that the i el labilities for both 
scales are high, over 80, and the coiielations between scales are 
comparably high In both scales, childien with low I.Q Lend 
to gam more upon letcsts than do the children of average I Q 
while those of above-average I Q actually tend to lose upon 
retesting For both tests, as the interval between siiccesBivc 
administrations increases, so do the discrepancies between the 
test ratings For both scales, as nioic tests aie taken, the discrep- 
ancies between latei tests tend to be bmallei than those between 
earlier tests 

In the ouginal scale the mean of the abboliito deviations is a 
maximum in the middle age lange, foi the revised scale it de- 
creases with age 

For the original scale there is a small increase in net gain 
with increasing age In the revised scale there is a decrease In 
the amount of gam 

The dispeision o£ LQ 's and therefoie the discnminal ability 
of the test incieases with successive tests on the original scale; 
on the revised, however, the dispeision decreases 

The scatter on the revised scales is greater and reaches its 
maximum later than on the oiigmal scale, 

Inversions in basal year scores are more frequent in the 
revised scale while skips are more frequent m the original scale 
Basal year test groupings on either test do not represent the 
normal peifonnaiice of children of the corresponding age but 
rathei of children a year or two older. 
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THE PREDICTION OF SCHOLASTIC SUCCESS IN A 
COLLEGE OF MEDICINE 

DEWEY B STUIT 
University of Iowa 

The prediction of scholastic success in the professional colleges 
IS a major personnel pioblem and one of primary significance to 
the individual, the colleges and society as a whole Satisfactory 
achievement m the piofessional courses is the first step toward 
vocational success Unless an individual can perform satisfactoiily 
the work required for a professional degree, the question of ultimate 
vocational success need not be raised 

If the individual can be informed of his chances foi success in a 
professional college before he enrolls it should be of gieat advan- 
tage to him m terms of time and money saved if he should othei- 
wise fail At the same time it should encourage those who possess 
the necessary ability to make the saciifices which may be involved 
The net results should be a better adjusted individual and a more 
competent profession While these statements apply to all profes- 
sional colleges, they seem particularly peitinent to medicine because 
the period of training is long, the expense to the individual is con- 
siderable, and the welfare of society demands highly competent 
medical men The piesenl investigation was undertaken to throw 
some light on the problem of predicting success in this professional 
area 

Specifically, it was the purpose of this study to investigate the 
value of liberal arts grade point averages and certain aptitude test 
scores as predictive indices of success in first year medicine at the 
State University of Iowa ^ Because of the variations in grading 
standards at different institutions only those students who com- 
pleted all of their undergraduate woik at the University and who 
had complete records for one year of work in medicine were included 
in the study. Prior to 1938 standards foi admission to the College 
of Medicine required at least two years or 60 semester hours of 
work m an approved college of arts and sciences; aftei 1938 this 

*Thc writer wishes to express hla appreciation to Dean E M, Mac- 
Ewen of the State Univeiaity of Iowa College of Medicine for making 
available the basic date and to Mr C William Applegate, icsearch assist- 
ant in educational personnel, for his contribution to the statistical analyses 
made In the study 
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wss to tliico yoiirs oi 90 ticuicslci lioiiii A.lii-Q ii\ 19^8 the 

required grade point average in libcial arts work was raised fiom 
2,00- to 2.20 The restrictions impo'icd made it ncccs&ary to select 
students from entering classes as fai back as 1934. Tlie numbci and 
percentages selected fiom each class arc presented in Table I. 


TABLE I 

Numbei and Pciccntage of SUidcnls Selected fiom Vaiions 
Frcahinan Classes in the CoUdge of Medicine 
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*The total number enrolled in each cl.iss nu hides hludciUs coinplctiiig 
their liberal arts ^oik at other instilutioiio in whole oi in p<iit, those 
registered as fieshmen for moie thnri one year, and lliobc who withdrew 
in the course of the yeai, 

The predictive indices available for this group of 142 btu dents 
included Iowa Qualifying Examination scoics, Moss Mcclicnl ApU- 
tude Teat scenes, and grade point averages for liberal aits woik, 
The Iowa Qualifying Examination, adminisierod to all cntciing 
freshmen, consists of the Iowa High School Content Examination, 
Iowa Silent Reading Test, Iowa Mathemalica Aptitude Tost, and 
the English Training Examination A composite scoie, consislmg 
of a weighted raw score total, is computed for the gioup of four 
examinations and is used as the score in the Iowa Qualifying Ex- 
amination 

The purpose of this examination is to assist counselors in then 
advisory work with students and to predict the schola*^tic success 
of undergraduate students in various colleges and cunicula. The 
Moss Medical Aptitude Test is administered to applicants Cor ad- 
mission to colleges of medicine by the Ameilcan Association of 
Medical Colleges In considering the liberal arts woik, the total 
grade point average, the “required science" and "lolal science" 
grade point averages were studied separately Required science, as 
distinguished from total science, includes 32 hours of pi escribed 
courses, The specific subjects prescribed in the liberal ai ts ciimcii- 

point averages or point hour latios aie computed by conindei- 

ing A 4, B = 3, C = 2, D = 1, Fd = 0 
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lum aie inorganic chcmistiy through qualitative analysis, quantita- 
tive analysis, elementary oiganic chemistry, elementary physics, and 
biological science, usually zoology, 

The ciiterion of success m first-year medicine consisted of the 
student’s giade point average at the close of the academic year 
Makeups for subject conditions and mcompleles wore disregarded 
It was felt that the grade first assigned iii a course should be used 
because it represented a better appraisal of the student’s perform- 
ance in comparison with his fellows. The same piactice was fol- 
lowed in computing grade point averages foi the second yeai of 
work 

The law scores m the aptitude tests had been converted to per- 
centiles and were thus lecorcled It was assumed that these percen- 
tiles weie equivalent fiom year to year For computational purposes 
the percentiles were converted into hneai scoies by the use of 
Hull’s table 


Student Perfoimnncc 

The perfoimaiice of the students in the aptitude tests and hbeial 
aits work is shown m Table II The mean hncai scoic of 44 50 in 
the Moss Medical Aplilude Test is equivalent to a percentile scoic 
of about 40 on nation-wide noims Data were also available for 240 
additional students who did not meet all of the critciia used in the 
selection of the 142 students included m this study, It will be noted 
that the mean linear score for this group is slightly higher, but the 
range is almost identical In the Iowa Qualifying Examination and 
its sub-tests, the group is definitely superior as indicated by the 
mean linear scores, but the range is very wide, varying from the 
seventh to the niiiety-mnth percentile in the composite score, The 
mean grade point average of these students in liberal arts work is 
also definitely superior The average point hour ratio for the Col- 
lege of Liberal Arts is about 2 20, while the students in this gioup 
achieved a 2 60 average From these data one might conclude that 
the typical student who goes into medicine at Iowa is definitely 
superior in the Iowa Qualifying Examination and in his liberal arts 
work, but he may be somewhat below the average in the Moss Med- 
ical Aptitude Test. 

The second phase of the study was concerned with the relation- 
ship between the various predictive indices and scholastic success 
in first year medicine* The coefficients of correlation expressing 
these relationships were computed by the product-moment method 
and aie presented in Table III 
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TABLE II 

Student Performance m the Aptitudi; Teats 
ond h' 

Liberal Arta Work 


Predictive Index 

N 

M 

a 

Ronge 

Moas Medical Aptitude 

141 

44 50 

15 05 

Linear Score li 83 
Pcrcciitdc 3 96 

Moss Medkd Aptitude 

382* 

46 00 

16 05 

Linenr Score 0 87 
Pcrceiililc 0 98 

Iowa Qualifyiiig Exam . 

1 Compoaltc Score 

U2 

62 29 

IS 57 

Lmeor Score 21 91 
Pcrccnlllc 7 99 

2 High School Content 

142 

62 90 

15 85 

Linear Score 9 91 
PtrcenUlc 2-90 

3 Math Aptitude 

142 

62 65 

15 94 

Linear Score 21 91 
Percentile 7 99 

4 English Training 

142 

56 59 

15 65 

Linenr Score 9 91 
Pcrccnlllc 2 99 

5 Silent Reading 

lag 

58 75 j 

16 2S 

Linear Score 22 gi 
Pcrccnlllc 8 99 

Required Science 

Total Science 

142 

142 

2 62 

2 60 

453 1 
308 

1 lot 4.00 

I 721 4 00 

Total Libei al Arts 

142 

2 60 

376 ' 

1 81) 3>61 

Fresh Med P K R 

142 

2 36 

OtD 

0 53 3 84 

Fresh Med P H R 

382 

2 33 

643 

0 4J 4 00 

Fresh Med P.H R 

112t 

2 45 

524 

1 57 '3 84 

Soph Med PHR 

lUt 

216 

52B 

1 03 3 61 


A Bupplementary $tudy was made of 382 sludciitSi 
t Students dJ the group oE 142 who completed two bOcccbbivc yoaia 
$The 2 20 requirement was in cRect in 1938. Picvioua to 1930 this had 
been 1 5o, and was then raised to 2 00 A few students admitted in 1930 
did not enroll until 1934 Only one had a total grade point nvciugc 
below 2 00 


TABLE III 

Correlation of the Predictive Indices with the Criterion 



N 

r 

PEr 

The Iowa Qualifying Examination 




Composite Score 

142 

ogs 

1 056 

High School Content Examination 

142 

058 

056 

Mathematics Aptitude Test 

142 

los 

.056 

Enghab Training Examination 

142 

025 

056 

Silent Reading 'Teat* 

139 

075 

056 

Moas Medical Aptitude Teat 

, 142 

,226 

054 

Mob 3 MediCEil Aptitude Teafc 

382 

.316 

031 

Liberal Arta Grade Point Average^ 

Required Science 

1 

, 142 

419 , 

046 

Total Science 

142 

465 

045 

Total tiiberal ArU Work 

142 

449 

045 
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Inspection of Table III reveals that the Iowa Qualifying Exam- 
ination correlates very low with success in first year medicine, that 
the correlation between the Moss Medical Aptitude Test scores and 
the criterion is hardly significant and that the total science average 
IS most closely associated with success in medicine as measured by 
first year grades Examination of the acatter-diagiams provided 
several clues which may explain certain of the correlations In the 
Iowa Qualifying Examination only 27 students received linear 
scores below 50, hence seriously restricting the range of talent of 
this group As a result a majority of the students arc concentrated 
m the first and second quadrants, those in the second quadrant hav- 
ing received high scores in the qualifying examination but achieving 
below average in first year medicine Much the same picture is pre- 
sented for each of the sub-tests compiising the qualifying examina- 
tion The data suggest a critical linear score of 40 or 45 in the com- 
posite score of the qualifying examination, for only eight students 
With qualifying scores below a linear scoie of 45 succeeded in mak- 
ing a 2 00 average or bettei in first year medicine 

The scatter-diagrams of the Moss Medical Aptitude Test present 
a striking contrast to those of the Iowa Qualifying Examination. A 
significant proportion of the students who score low in the test do 
very well in freshman medicine As shown in Table II, the average 
grade in first year medicine is 2 36 and in the Moss test the mean 
linear score is 44 50, A total of 29 students or slightly over 20 per 
cent scored below aveiage in the Moss test, but made grades above 
2 40 in first year medicine The student scoring lowest in the apti- 
tude test succeeded in making a 2 60 grade point average Poor 
performance in the Moss Medical Aptitude Test does not appear 
to indicate with a high degree of certainty that the student will do 
poorly in medicine at this institution The scatter-diagrams for the 
382 students present a similar picture 

Of the indices computed from the students’ liberal arts records, 
the total science grade point average correlates best with scholastic 
success in medicine However, there are some extreme deviates who 
reduce the magnitude of the coefficient of con elation, For example, 
one student with a 2 00 liberal arts record made a 3 35 grade point 
average in medicine while another with a 3 00 record in liberal arts 
work made only a 1 SO average in medicine In general, however, 
there is rather close agreement between the grades in the two cur- 
ricula. It does not appear that the liberal arts science record is 
superior in predictive capacity to the student's general average m 
undergraduate work. When the total science average and Moss 
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Medical Aptitude Test scores aie combined as predictive indices, 
the multiple correlation is <^94 Apparently the Moss test does not 
add greatly to the predictive capacity of the science giades. 

Supplemeiitaty evidence concerning the relation between the 
Moss Medical Aptitude Test scoies and liberal aits grade point 
averages on the one hand and success m medicine on the other is 
furnished m Table IV, With one exception the mean giade point 
average in libeial aits woik mcieases as the level of achievement m 
medicine increases Tins is also tiuc of the mean score m the Moss 
Medical Aptitude Test but the tiend is not as pronounced It is 
interesting to note that the range of performance as regaids pre- 
dictive indices is about the same foi all Icveib of achievemciu This, 
makes the low correlations less surpiising 


table IV 

Moaa Medical Aptitude Test, Mean Lincai Scores and Libeial Arts Mean 
Grade Point Aveiagca for Varioiis Levels ol Achievement 
m Frcshtnaii Medicine 


Fieihniftn ' 
Medlce 

MoflB AptlcoOe 

Required Silence 

Tolnl Stlcncc \ 

PHR 

N 

M 



M 

cr 

RnhBC 

M 

a 

Uaii( c 

3 OQ 4M 


si'i? 

14 9a 

22 B3 

2 03 

SI 

2 00 4 QO 

2 52 


2 00 4 00 

3 SO 2 9!) 

34 

40 59 

14 61 

13 73 

2 66 

39 

1 ea 318 

261 

34 

2 00 3 22 

2 00 2 

42 

47 60 

13 OS 

10 71 

2CS 

40 

1 63 3 So 

2 63 

28 

2 00 3 22 

1 so 1 09 

30 

40 28 

13 22 

IS 68 

2 37 

3S 

1 SO 3 00 

2 32 

33 

1 72 3 07 

0 QO 1 49 

12 

3717 

is 92 

1 10 70 

2 25 

la 

1 75 2 75 

2 29 

Ifi 

2 DO 2 Cl 


Tolnl L A Work 

M 

a 

Rnnia 

2 (J3 

3^ 1 

2 16 3 61 

2 60 

2 1 

2 II 3 12 

2 65 

22 

2 05 a so 

2 35 

1i 1 

1 01 3 31 

2 28 

16 

2 00 2 58 


TABLE V 

Student Persistence m the College of Medicine 
at the State University of Iowa 


Ye^kC Enteved 

One Year 

Two Ycr\s 

Three Years 

Tour YcaiB 


N 

N 

N 

N 

1934 

34 

30 

29 

27 

1933 

40 

35 

32 

32 

ld3fi 

21 

19 

17 


1937 

33 

2B 



193S 

1 14 

, 

1 



In order to ascertain whether generalizations made concerning 
first year medicine would apply to other years, the freshman and 
sophomore grades for IIZ students V(?ere correlated The resulting 
coefficient was 722, It also seemed dcaltable to know i£ the students 
who complete one year of work continue beyond that point. The 
results are presented in Table V and seem to warrant the conclu- 
sion that the persistence of students beyond the freshman year is 
very high It also appears that the first year’s work is strongly 
indicative of later success m the medical school 


82 



PREDICTION OF SUCCESS IN A COLLEGE OF MEDICINE 


The results of the piesent study agree very well with those found 
at other institutions m this region At Minnesota'* the correlation 
between Moss Aptitude Test scores and fieshman honor points was 
found to be 27 foi the class of 1938 and 22 foi the class of 1939 
Liberal aits grades for these same classes showed coi relations with 
freshman grades in medicine of 57 and 46 respectively In a study 
made of the classes enteiing the Univeisity of Illinois* in 1932 and 
1933 the correlations between libeial ait^ aveiages and achicvcrneiit 
in first yeai medicine weic found to be 49 and 41 lespectivcly 
Comparable coirelations foi the liberal aits aveiage in science weie 
57 and 42 The Moss Medical Aptitude Test was administered to 
the class entering in 1932 and con elated to the extent of 42 with 
first year medicine Not all the reports on the prediction of success 
m medicine published in the Jotirnsl of the Association of Amci^ 
lean MedicaJ Colleges are in agieement with these findings Some 
repoit the Moss test as being superior to the liberal arts giade 
point average in piedicting success in medicine while others find 
the reverse to be true Apparently all agree, howevei, that aptitude 
tests and the undergiadiiate grade point averages furnish informa- 
tion which IS valuable m selecting students foi medical colleges 

Conclusions 

The data seem to wairant the following conclusions foi the pop- 
ulation included in this study oi populations which aic similar 

1, Libeial aits giade point aveiages are the best pieclicUve 
indices of success in first yeai medicine Rcquiied science, total 
science and total libeial aits woik aie of about equal value in this 
respect 

2, The con elation between the Iowa Qualifying Examination 
scores and grades in fieshman medicine la very low However, the 
data do suggest a cntical score which might be used by counselors 
in then advisory work with students who are interested in medicine 
as a career 

3 In this institution the Moss Medical Aptitude Test does not 
predict the student’s level of achievement with high precision, Stu- 
dents scoring low in the test may do very well in mecUcine. 

‘*J W Cavete, A. T Heiiiici, niid S B. Lmcllcy “TestH of Medical 
Aptitude at Minnesota" Journal of the A^>\ociation of Aineiicnn Medical 
Colleges, XII (Septembei, 1937), 257-68 

* George R Moon "Study of Premcdical and Medical Scholastic Rcc- 
oids of SUuleiita m the University of Ulmoia College of Medicine" 
Journal of the Associaiion of Amertcau Medical CoHege^, XIII fi93B) 
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4 To predict success in medicine with greater accuiacy will 
require tests which distinguish more clearly between various levels 
of ability It is also possible that the criterion of success will need 
to be defined more precisely 

5, The counselor should not rely solely upon aptitude test 
scores and grade point averages in advising students about their 
probable success in medicine Pei haps average achievement in apti- 
tude teats and libeial arts work pl\is high interest and motivation 
will insure the student's scholastic success. 
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GIVEN AT THE UNIVERSITY OF CHICAGO-*' 

A COMPARATIVE STUDY OF FRESHMAN WEEK TESTS 

WILLIAM M SHANNER 
Civtl Aeronautic^ Authority 
and 

G FREDERIC KUDER 
Sociaf Sccurjfy Roarrf 

One of the most crucial problems that confioiits the educator 
of today is that of correctly advising students as to their education 
and vocational careers In conjunction with this problem, educa- 
tors and psychologists have constantly worked to secure more 
reliable and accurate information for use in counseling students 
Almost every college and university now has an orientation week 
at which time all incoming students are required to take batteries 
of psychological and placement examinations, the lesultB of 
which are used in advising students relative to their educational 
programSi 

In September, 193B, a compiehensive battety of psychological 
and placement tests was administered the incoming freshman 
the relationship between these tests and succeeding academic 
class at the Univeisity of Chicago with a view toward studying 
achievement at the university Among the tests administered the 
freshman group were the sixteen sub-tests of the American Coim- 
cil on Education Tests for Primary Mental Abilities, Expeii- 
mental Edition, the 1938 Form, College Edition of the American 
Council on Education Psychological Examination, the College 
Entrance Examination Board's Scholastic Aptitude Examination, 
a physical sciences aptitude test, a social sciences aptitude test, 
Pressey’s Special Reading Test, Form A; Pressey's Test on Read- 
ing Comprehension, Form A, and a vocabulary test The physical 
sciences aptitude, social sciences aptitude, and vocabulary tests 
were locally constructed 

The first two years of the University of Chicago arc devoted 
to a program of general education, The curriculum includes four 
introductory survey courses in the following fields! biological 

*This study waa made while the wnteis were with the Board of 
Examinations at the UrLiversity of Chicago 
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sciences, humanities, physical sciences, and social sciences, and a 
number of elective second-year or sequence courses in specific 
subjects The typical student takes two of the foiii general 
courses during his freshman yeur and the two remaining courses 
during his sophomore year His progiam each year is completed 
by one or more of the other courses offered in the college, The 
general courses extend throughout the school year, and achieve- 
ment is measuied at the close of the ycai by means of a six-hour 
comprehensive examination Attendance at a course is not re- 
quired, The only requirement is the successful passing of the 
comprehensive examination. Many students are advised, upon the 
basis of their performances on the freshman week examinations, 
to attempt a comprehensive examination without taking the 
course. Since all students entfiung the Univeislty of Chicago 
as freshmen are required to take the foui general courses, educa- 
tional advisers are confionted with the pioMem of selecting the 
most appropriate general courses for the ecliicntional piograin of 
the student and advising him as to whethei he needs additional 
assistance or should attempt the comprehensive oxammation with- 
out taking the course 

By June, 1939, 501 of the ficshmen enleiing the Univoisity in 
September, 1938, had taken one or moic of the comprehensive 
examinations for the four survey courses and various sequence 
courses The grades of the comprehensive examinations are re- 
ported m terms of derived scores having a mean of 20 and a 
standard deviation of 4 The average examination grade of each 
student was found by adding the derived scores for all hia com- 
prehensive examinationji and dividing by the number of 
examinations 


Test Scores and Average Grades 

Table I reports the correlation betv/een the various freshman 
week tests and average examination grades. The testing time of 
each examination is also reported The social sciences aptitude 
test has the highest correlation with average grades (.575), The 
physical sciences aptitude, the American Council Psychological 
Examination, and the College Entrance Examination Board's 

’•'For a comprehensive description of the organi^ntion of tho first two 
years of the Univeralty of Chicago, see Chauncey Samuel Boucher, and 

r J The Chicago College Pkn (Chicago- The UnlveiaLty 

of Chicsgo Press, June, 1940), pp xuU4l3, 
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Scholastic Aptitude Examination have just slightly lowei corre- 
lations All these tests, with the exception of the CEEB Scholas- 
tic Aptitude, require appioximately one hour of testing time 
each, the CEEB examination requires two hovns’ time 

The social sciences aptitude test is essentially a reading test 
It consists of three selections, one each drawn fiom the fields of 
economics, sociology, and political science, Each paragiapli is 
followed by a number of questions based upon an understanding 
of the materials covered, The test is a revision of a test given 
experimentally the pievious year. 

The physical sciences aptitude test consists of (1) a section 
on vocabulaiy in the field, (2) qiieations involving the interpre- 
tation of mathematical formulas, and (3) a leading test containing 
chemistiy and physics selections It is the product of a process 
of analysis and revision carried out ovei a peiiod of yeais. 

The results of the 16 tests of the Piimaiy Mental Abilities 
battery are reported in terms of seven composite scores, each an 
approximation to a factor Scoies for the following abilities nre 
reported foi the test 

Perceptual This ability, measured by the verbal enumeration 
and identical forms tests, may be described as one's facility in 
finding detail which is significant to him or detail which he is 
seelring 

Number, This factor consists of facility with simple numer- 
ical work and is measured by the tests of rapid addition and 
multiplication, 

Verbal, The verbal factor manifests itself in the completion 
and in same or opposite tests It is roughly the ability to deal 
readily and quickly with verbal materials, 

Spatial The spatial factor is measured by tests requiring the 
subject to think visually of geometric foims and of objects in 
space, 

Memory This factor is one’s ability to memorize various mate- 
rials. One test requiring the memorization of initials with names, 
and a second test requiiing the association of words with num- 
bers are used m measuring the ability 

Inductive Reasoning, The induction factor may be described 
as one's ability to discover some rule or principle in various 
arrangements of material A numerical, a verbal, and a spatial test 
are used in estimating the ability 
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DeduQtivo Reasoning, The deductive factor may be described 
as facility in formal reasoning It is measured by tests of arith- 
metic problems, numbei senes, and perception of mechanical 
movements 

The largest correlation between a composite primary ability 
score and average grades is for the verbal composite (415) which 
requires 16 minutes of testing time. The remaining coi relations are 
much lowei These results are reasonable, since the composite 
scores represent specific abilities, and aveiago grades represent 
general academic proficiency, 

Test Scores and Course Grades 

Table II reports the correlation between various fieshman week 
tests and grades for the foin general courses The two largest cor- 
relations (.654 and ,6'I8) are for the physical and social sciences 
aptitude tests with the respective geneial coiiises, The coirelatious 
for the American Council Psychological and the CEEB Scholastic 
Aptitude Examination show no statistically significant difference 
for the biological sciences, social sciences, and physical sciences 
general course examinations However, the coi relation between the 
CEEB and humanities is significantly gi eater than between hu-* 
manities and the American Council Psychological Examination. It 
IS of interest to note the variations in the si/e of the con elation 
coefficients for the composite scores of the Primary Mental Abilities 
battery. The two largest correlations for the composite scores are 
between Deduction and grades in the physical sciences, and between 
Verbal and grades in the humanities One might very well expect 
these phenomena At the same time, humanities shows a coi relation 
of only ,071 with the composite Spatial score. 

The degree of independence of the seven composite scores of the 
Primary Mental Abilities battery is reported in Table HI, which 
gives the intercorrelations for the seven scores Slightly over half 
of the correlations m the table are less than ,300 and two can be 
considered as zero, These small correlatioris suggest a considerable 
degree of independence for these scores and that they might well 
measure specific abilities. The intercoireiations among Spatial, In- 
duction, and Deduction scores are all very near ,500 and thus give 
evidence of considerable dependence of scores, 

The first line of Table IV reports the multiple correlation co- 
efficients for the combination of the two primary ability composites 
having the highest validities with respect to each of the four general 
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courses The Veibal and Deduction scores, from tests requiring 70 
minutes, were combined for all except the humanities course, For 
this course the Veibal and Number scores, from tests requiring 35 
minutes, were combined, The second line of Table IV reports the 
multiple correlations obtained by combining all seven scores of the 
Primary Mental Abilities Tests These coejficients are not markedly 
higher than those obtained from the best two in each case 


TABLE I 

Testing Time Required tor Administering Vaiious Psychological and 
Placement Tests to the 1930 Fieshman Claea at the University of Chicago 
and the Correlation of these Tests with Average Grades 



Tcallng 
Tlnw la 
Mlnutea 

Correlation 

vilUi 

Average 

Oradca 

Test 

TtntUift 
TLmfl ivi 
Mlniilci 

CorrcUllon 

* AvcraiEc 
Grnilcfl 

A 

Perception* 

20 

117 

American Council Psy- 
chological Exominalion 

56 

523 

Number* 

19 

310 

College Entrance Examnm 
lion Board's ScholaBlic 
Aptitude 

120 

542 

Verbal * 

16 

415 

Physical Sciences Aptitude 

60 

522 

Spatial* 

33 

1B4 

Social Sciences Aptitude 

60 

575 

Memory’* 

33 

204 

PrcBsey's Special Reading 

60 

477 

Induction* 

1 

229 

Prcsaey'a Reading Conipre 
hensjon 

t 

326 

Deduction* 

54 

378 

Vocabulary 

25_ 

486 


'f' Composite scores for the Thurston e Testa for Primary Mental 
Abilities 

t Specific time limits are not given*, the students ate given the time 
necessaiy to read entire reading selection and answer the queBtlolis Ap- 
proximately 20 minutes are icqulred. 
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TABLE II 

CoeHicieiits of Con elation between Vaiioiia PbyclioIOBical and Place* 
ment Tests and the Four Introductoiy General Comtes at the University 
of Chicago 


Teat 

Biological 

Sciences 

Humani 

tics 

Physical 1 
Sciences 

Social 

Sciences 

Percept] on ^ 

DS4 

,12!) 

166 

1 135 

Number* 

207 

1 ,265 

272 

300 

VerhaU 

jao 

472 

d76 

43S 

Spatial* 

225 

071 1 

139 1 

131 

Memory* 

145 

127 

177 

160 

Induction* , 

2ia 

029 

247 

196 

Deduction* 

413 

,190 ' 

485 

427 

American Council PsycUobgicnl 
Examination 

433 

.4B5 

482 

569 

College Entrance. Examination 
Board's Scholastic Aptitude 
Examination 

479 

544 

471 

577 

Physical Sciencea Aptitude 

— 

— 

054 

— 

Social Sciences Aptitude 

— 



— 

61 B 


Composite scoica for the Thurbtone Testa fov Piimary Menttil 
Abilities. 


TABLE HI 

Intercorrelationa for the Seven Composite Scores for the Thnrstono 
Tests for Primary Mental Abilities 



u 

i 

a 

Verbal 

13 

<-f 

+J 

P« 

0 

, 3 

s 

1 § 

' s 

1 1 

M 

f’ 

Deduction j 

Perception 

237 

171 

1 .392 

046 

35S 

153 


Number 

250 

204 

IdBi 

306 

.336 



Verbal 

21S 

156 

347 

368 


1 


Spatial 

077 

.490 

' ,475 


1 



Memory 

.170 

1 126 







Induction 

1 533 
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TABLE IV 

Multiple Coriclation Coefficients between Various Combinations of the 
Composite Scores of the Primaiy Mental Abilities Tests and the Four 
Introductory General Courses 


Conibinalion of 

Composite Scores 

Biological 

Sciences 

1 

HuiTtani- 

ties 

Phyaicnl 

Sciences 

Social 

Sciences 

Two Best Piedicting Composite 
Scores 

'184 

496 

,529 

521 

All Seven Composite Scores 

500 

S41 1 

561 

556 


Conclusion 

Two rather striking observations may be made on the basis of 
the results reported 

(1) Marks m the four couisos can be piedicted by combining 
two fairly short primary abilities measureB about as well as 
by using the one-hour American Council Psychological Ex- 
amination or the two-hour scholastic aptitude test of the 
College Entrance Examination Board, both of which were 
constiucted for the purpose of predicting scholarship This 
result IS the more remarkable since the Piimary Abilities 
Tests were not specifically constructed for the purpose of 
predicting giades 

(2) Tests developed for the specific situation are m the present 
case more efficient prognostic measures than any other 
single test or combination of tests studied The validities 
of the aptitude tests for the physical sciences and the social 
sciences are significantly higher than the other validities 
obtained, 

These two results appear to be essentially contradictory One 
of them argues for the development of a number of relatively inde- 
pendent measures and the use of those which, in combination, are 
most efficient foi the prediction of any selected criterion or group 
of criteria The other seems to indicate that tests constructed and 
revised in the light of analysis with respect to the local situation 
are most effective, at least when compared with two of the bcttei 
scholastic aptitude testa constructed for general use, This conclu- 
sion IS valid for the tests studied in their present state of develop- 
ment However, it is apparent that what can be measured in com- 
posite tests, such as the aptitude tests in the fields of the social and 
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physical sciences, can be measured by a number of moie specific 
and relatively independent measures The difference between the 
predictive efficiency of the primary abilities measures used as com- 
pare d with the specific aptitude tests must be attributed to the fact 
that the former do not sample some of the attributes included in 
the latter The fact that the Primary Mental Abilities Tests in 
combination produce fairly high validities although they were not 
developed for the purpose of predicting scholarship is indicative of 
the promise in this type of measuiement As the experimental tests 
of primary abilities are perfected and expanded to include other 
abilities involved in scholastic success, it is reasonable to expect 
combinations of them to approach and equal the validities of tests 
constructed for each of a number of specific situations This devel- 
opment will make practical a much more efficient use of test mate- 
rial when a number of criteria are to be predicted 
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NOTE ON A SIMPLIFIED METHOD OF COMPUTING 

TEST RELIABILITY 

C J HOYT 
University of Mfimesotn 

Kuder and Richardson^ have picsented the theoretical bach- 
ground as well as useful formulas for a new and improved proce- 
dure for estimating the coefficient of test reliability. In a later 
paper® they have labeled their procedure “the method of rational 
equivalence ” Then results appear to have a number of important 
advantages over the split-half correlation method used in con- 
junction with the Spearman-Brown formula With the split-half 
method the obtained coefficient may be an ovei estimate or an 
underestimate of the actual leliability With the method of 
rational equivalence the estimate derived is known to be nevei 
an overestimate ' This fact alone is sufficient for lecommeuding 
the displacement of the split-half procedure, although there aic 
other advantages, as pointed out below 

The theoretical soundness of the Kuder-Richardson deuvation 
IS indicated by the fact that analysis of variance techniques 
applied to this problem produce an identical formula The present 
writer’s derivation, using an approach entirely diffeient from that 
used by Kuder and Richardson, will appeal elsewhere, 

The use of the formula recommended by the authors for gen- 
eral use requires only the same piiraary data as are ordinarily 
obtained in a careful analysis of a test Consequently, it is not 
necessary to obtain the scores on separate pacts of the test. The 
possibility of obtaining varying results with different methods 
of dividing the test is also obviated The compulations it\volved 

’G F Kuder and M.W Richacdaoii “The Theory ot the Estimation of 
Test Reliability" PsychometnkA, II (1937), 151-60 

“ M W Richardson and G F Kuder “The Calculation of Test Re- 
liability Coefficients Based on the Method of Rational Equivalence'’ 
Journal of Educational Psychology^ XL (1939), 681-87, 

^Tliis statemeiit is strictly tiue for the population used. Sampling 
errors are, of course, not eliminated For a discussion of sampling errors 
the reader is referred to Robert W Jackson, “Reliability of Mental Tests " 
British JourtiRl of Psychology, XXIX (1939), 267-87 
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in computing the coefficient of leliability by the method of 
rational equivalence can be perfoimed in a few simple steps that 
do not require any special statistical knowledge 

The increasing use of the method o£ lational equivalence for 
the estimation of test leliability leads the wiitei to deaciibe a 
procedure which he has found to be paiticulaily efficient Al- 
though Knder and Richaidbou pieseiit a number of formulas 
involving various degrees of rigor, they lecommciul then formula 
(20) foi general use, Their empiiical findings and those of a num- 
ber of others who have been using the method indicate that the 
results obtained from their foimula (20) closely approximate 
those obtained by the moie rigorous foimulas The steps outlined 
below have therefoie been developed foi a variant of the lecom- 
mended formula/ 

1 Score the tests for the number of light answers Obtain 
the sum of these scores for all the subjects This value 
IS T in formula (1) below, 

2 Square each of these scores and obtain the sum of these 
squares for all the subjects This sum is Ss m the foi- 
mula below 

3, Make a tally of the test responses to each item and 
obtain the count of the numbei coirect for each item 
The total of these counts should equal the T obtained 
in step 1, 

4, Square the count obtained for each item and obtain the 
sum of these squares for all the items This bum is Si 
m the formula below, 

5 Using the values obtained in the steps above, solve the 
following formula for r^, the leliabiUty of the teat In 
this formula, k is the numbei of subjects taking the test 
and n is the number of items in the test 


n kSs+Si — T(TH-k) 

( 1 ) 

n-1 kSs — 


In the use and analysis of a test some of these steps will 
already have been performed. Use of the item counter of the 
International Scoring Machine will greatly facilitate step 3 If 


‘While the present paper was in piesB, Paul L DicBsel published other 
variants of the Kuder-Richardaon formulaa Formula <1) above vs cquiv- 

^ Crease], "Some Renvaiks 
R«hability Coefficient Psyohemetnka, V 
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a computing machine is not available, steps (2) and (4) can be 
greatly facilitated by the use of a table of squares and an or- 
dinary adding machine 

The use of foimula (1) will be illustrated m a particular ex- 
ample involving a test of 250 items administered to a group of 
33 students in the College of Phaimacy at the University of Min- 
nesota The values obtained m this case were as follows 

Si = 112,873 T = 4629 Ss=^727,351 

r„__250 33(727,351) -\- 112,873 — 48290829 -| 33) 

249 33(727,351) — (4829)= 

250 C36,858 159,214.500 

— - — 1 Q3fi 

249 603,342 170,152,158 

Foimula (1) is algebraically equivalent to foiintila (20) pre- 
sented by Kuder and Richaidson Then formula (20) is as fol- 
lows ' 


n at" — npq 

ru = - 

n-i 0,2 


n o," — 
n~J Gi" 


where (n is the standard deviation of the dislnbulioii of test 
scores, pi is the proportion of students passing each item taken 
in turn, and qi is the pioportion failing that item, 

It should be remembered that this procedure is no more ap- 
plicable to speed tests than is the Spearman-Brown foimula. 
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Bedell, Ralph "Scoring Weighted Multiple Keyed Te&ta on the 
IBM Counting Sorter -Psyc7JO/2Je^^lAaJ V (1940), 195-201 

Tests or personal inventories with diffeiential item response 
weights may be scored by means of punch card equipment. De- 
tailed instructions are given for preparing the cards and scoring 
the forms The scoring speed u, approximately four to eight times 
that attained by manual scoring (Courtesy Psyc/iomefn/raJ 


Blakey, Robert, "A Re-Analysis of a Test of the Tlieoiy of Two 

Factors “ Psychometrika, V (1940), 121-36 

The study of William Brown and William Stephenson, "A 
Test of the Theory of Two FacLois," is ce-analy.<red by means of 
the Thurstone multiple factor methods No tests or correlations 
are left out of the original table of correlations as is done m the 
original analysis in an attempt to validate the two-factor theory, 
Space, veibal, and perceptual speed factois similar to those found 
by Thurstone, Wright, and Garrett are identified A common 
factor of "Maturation" is postulated to account for the remain- 
ing communality of the tests A fifth factor is considered to 
have no significance due to the small amount of variance which 
it contributes to the total, (Courtesy Psychoiuetnka ) 


Blum, M, L, "A Contribution to Manual Aptitude Measurement 
m Industry the Value of Certain Dexterity Measures for the 
Selection of Workers in a Watch Factory” /oiirnaf of Ap- 
plied PsyeWogy, XXIV (1940), 3B1-416 

Job analysis of watch assembling suggested the importance of 
the ability to make fine finger movements, the ability to handle 
tweezers, and the ability to continue to perform delicate tasks 
without increasing tension or maladjustment Three criteria of 
proficiency were established' length of employment, salary ratio, 
and foremen‘s ratings Two hundred and fifty-eight women (37 
workers, 137 applicants before being hired, 84 applicants aflei 
being hired) were examined with the O’Connor Finger Dexterity 
and Tweezer Dexterity tests Time scores showed the highest 
prediction of the proficiency criteria. The practical value of crit- 
ical time scores on the dexterity tests was indicated W A, 
VarveU 
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Cattell, R B “A Culture-Free Intelligence Test Pait I Journal 

of ^ducationa/ Psychology y XXXI (1940), 161-79 

A common source of error in the Binet-Simon type of test 
arises from the influence of academic expeuence and general 
cultural hackgroimd Instead of sampling the “common knowl- 
edge” of the subj[ect, the test emphasizes the perception of rela- 
tions inheient in objects and piocesses common to a wide range 
of cultuial groups One hundred multiple choice scaled items are 
chosen from ma^es, aeiies, classifications, progiessive matrices 
(3 types), and miiror images. The progressive matrix test con- 
sists of combined analogy and piogies&ive aeiies itemsi Harold 
Bechtoldt, 


Dunlap, Jack W “Problems Arising from the Use of a Separate 

Answer Sheet,” Journal of Psychology, X (1940), 3-48. 

The use of a separate answer sheet has been considered in 
teims of validity and leliahility of the moie conventional type 
of response Underlining, marking parentheses, maiking sep- 
arate answer sheets using serial and lepetitive numbering for 
choices with the answer sheets of both aiticulated and non- 
articulated types lead to piactically identical results Com- 
parisons were made in terms of means, standard deviations, i el la- 
bilities, and validity of lest lesuUs foi both fourth- and eighth- 
grade pupils The use of an aiticulated, seiial numbered answer 
sheet IS recommended foi tests shoit enough for all answeis to be 
recorded on a single side of the sheet Harold Bcchtoldt 


Hanell, Willard “A Factor Analysis of Mechanical Ability 

Teats” Psychomefrika, V (1940), 17-33. 

The intetcoiielations of 37 vauables, including the Minnesota 
battery of “mechanical ability’’ tests, the seven MacQuauic tests 
of “mechanical ability,” O’Connoi’s Wiggly blocks, and the Sten- 
quist picture-matching test, were analyzed by Thurstone's cen- 
troid method Five factors, Peiceptual, Verbal, Youth, Manual 
Agihty, and Spatial, were taken out Factors prominent in so- 
called mechanical ability tests are the Spatial and Perceptual 
ones with MacQuaine’s dotting test significantly high in the 
Manual Agility factor, Each of the factors can be measured with 
group pencil-and-papei tests (Coiutesy Psychometrika,) 


Harrell, T, W, and Faubion, R W “Selection Tests for Aviation 
Mechanics,” Journal of Consulting Psychology, IV (1940), 
104-05, 

Students of the United States Army Air Corps Technical 
Schools take a basic course of Shop Mathematics, Mechanical 
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Drafting and Blueprint Readmg> An Corps, Fundamentals. 
Metal Work, and Electucity besides specialiijing in some field’ 
Correlations of 38 tests with these five basic courses range from 
— 20 to + 54. Four tests give a multiple coi relation of 72 with 
a composite basic giade A factor analysis is being made of 24 
of these variables Harold Becbtoldt 


Johnson, A P, “A Btudy of One Company’s Ctiterm foi Selecting 
College Graduates" Journal of Applied Psychology, XXIV 
(1940), 2S3-64, 

A company had foi some ycais considered applications foi 
sales positions (personnel, advertising, and sales piomotion) on 
the basis of an intelligence test, a vocabulary test, and ratings on 
family background, mdustriousnesB, extioversion-introversion, 
and flair for writing, The piesent study examines the data for BO 
applicants (41 hired, 39 rejected) and seeks to objectify the rat- 
ings and to establish estimates of then reliability and validity 
The combined ratings of six members of a class in industrial psy- 
chology showed satisfactory leliability Ratings on "writing 
flair" most markedly differentiated the hiied from the rejected. 
Ratings on "family background" showed the highest correlation 
(4-0 40 ± 0 12) with BCivice oi ment ratings made by five com- 
pany executives on 23 workers W, A VarveJ 


McCloy, C H "The Measurement of Speed m Motoi Peiform- 

ance" Psychometrika, V (1940), 173-82. 

When the centroid method of factor analysis was applied to 
two sets of data on athletic performances, three significant fac- 
tors emerged strength, velocity, and dead weight, Scores on this 
speed factor were predicted by the multiple regression technique, 
the factor loadings on the speed factor being used as the ciiterion 
correlations, and these predicted scores were correlated with each 
of the other variables When the oiiginal tables, augmented by 
the new speed variable, were refactoied, the computed speed fac- 
tor fell on the speed axis as a primary trait It is thus shown 
that it is possible to isolate and measure a facloi which appeals 
in variables under consideration only as a compound, (Courlesv 
PsycftoinetriJta ) 


Palmer, C E , and Klein, H "A Table of the Double Integral of 
the Gaussian Probability Function" Child Development, XI 
(1940), 61 -8 F A Kingsbury 
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Roslow, S, Wnlfeck, W H„ and Coiby, P G ‘'Consumer and 
Opinion Reseaich Experimental Studies on the Form of the 
Question " Journal of Applied Psychology, XXIV (1940), 
334-46 

Summaries of the lesiilts of studies on vaiyiiig the foiiii 

of questions aie given Alteinate forms of the questionnaire, 
successive foims one month apaitj and fiee i espouse questions 
were among the methods used The use of steieotypcs or emo- 
tionally charged words pioduced significant changes in responses 
Slight changes in wording may oi may not lesull m changes in 
frequencies of the response choices, The completeness and num- 
ber of alternatives offeied in check lists tend to influence the 
proportions for any one response* while the results fioni free- 
response questions may be definitely misleading Haro/d Bech- 
toldt 


Sarbin, T R, and Berdie, R F "Relation of Measured Inteicsts 

to the Allport-Vernon Study of Values" Journal of Applied 

Psychology, XXIV (1940), 287-96 

Fifty-two university students were given the Allport-Vernon 
Scale and the Stiong Vocational Interest Blank, Form M A 
modification of the pattern analysis desciibcd by Darley was 
applied to the Strong piofiles Occupational keys were grouped 
accoiding to the results of factor analysis studies, "A few of the 
occupational groups showing measured inteiest patterns aie char- 
acterised by certain profiles on the Allport-Vcinon Scale." Al- 
though there is considerable oveilapping between gioups, "it is 
possible, nevertheless, , to use the Allport-Vernon Scale to 
approximate certain occupational interest types as measured by 
Strong, Thus, a definite but limited use is demonstiated for the 
Allport-Vernon scores when it is desirable to distinguish oi 
identify vocational interest types in the piofcssional, sales, or 
'uplift* occupations ” W A.Varvel 


Schult 2 , R S "Preliminaiy Study of an Industrial Revision of 
the Revised Minnesota Paper Foim Board Test'* Journnl of 
Applied Psychology, XXIV (1940), 463-67 

The Likert-Quasha Revised Minnesota Papei Foim Board 
Test (Form AA) was furthei levlsed foi industrial use in order 
to decrease the demand on veibal comprehension of the instruc- 
tions and to simplify the response required A preliminary study 
of this industrial levision la reported, Correlations with the 
Revised Minnesota range from -|-.71 to -| 86. Scores on the indus- 
trial revision tend to be significantly higher Twenty-one engi- 
neering students obtained a highei aveiage score than did 42 
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trade-school boys and 57 higb-school girls Correlations with 
intelligence correspond with those found in previous studies with 
the Paper Form Board Test, W A VarveJ 


Seder, M “The Vocational Interests of Professional Women" 

Part II Joiirna} of Applied Psychology, XXIV (1940), 265-72 

Sixty women physicians and 69 life insurance saleswomen 
filled out both the men's foitn and the women's form of the 
Strong Vocational Interest Blank Foi the 268 items common to 
the two forms, the median numbei of discrepant responses was 
18 per cent, so that the test-retest reliability is considered satis- 
factory The common items are as heavily or moie heavily 
weighted than items occiining on only one blank In general 
there is substantial agieement between the weights assigned to 
the response to each item by the men’s key and by the women’s 
key for the same occupation “All indications of this study are 
that differences between sexes m an occupation are usually less 
frequent and less impoitant than similarities" It is suggested 
that a common blank should be composed and that where sex 
differences actually appear an occupational key for each sex 
should be constructed W. A Varvel 


Thurstone, L L “Experimental Study of Simple Structure " Psy- 
chometrikaiV (1940), 153-68 

A battery of 36 tests was given to a group of high-school 
seniors The factorial analysis leveals essentially the same pri- 
mary factors that were found in previous studies The test bat- 
tery reveals a simple stiucture (Couitesy Psychometrikn ) 


Tucker, Ledyard R “The Role of Coirelated Factois in Factor 

Analysis " Psychotnetnka, V (1940), 141-52. 

The fundamental facto i theorem is developed in matrix form 
for the case of correlated factors The properties of the corre- 
lated factor system are discussed, and some effects of sampling 
error considered The psychological meaning of correlated fac- 
tors is discussed, and several mechanisms by which general fac- 
tors may operate m the factorial system are indicated. (Courtesy 
PsychowetTika.) 


Walker, Helen M “Degrees of Freedom." Journn] of ISducatioJial 
Psychology, XXXI (1940), 253-69 

The number of degrees of freedom is a basic concept in small 
sample theory Most textbooks omit a discussion of this topic, 
and many texts give incorrect formulae and procedures because 
0 ignoring it The development starts with the freedom of move- 
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meiit of a point in space undet certain lestraming conditions and 
utilizes the repiesentation of a statistical sample by a single point 
in N-dimensional space Illustiations are presented showing how 
to determine the niimbei of degrees of freedom appropriate foi 
use in ccitam common situatioin, as standard eiroi of the mean, 
Chi-sqiiare test, contingency tables, paitial coii elation, and anal- 
ysis of vai lance formulae Harold Bechtohit, 


Young, P V "The Validity of Schedules and Qiicstionnaues/’ 

Joiiinal of Educational Sociology, XIV (1940), 22-6 

A brief summary is given of an expeiiment with a variety of 
questionnanes and schedules as used on thice consideiably homo- 
geneous communities Objective, quantitative data were difficult 
to obtain The data shed little light on complexities of social 
patterns or on behavioi patterns of cultural worlds in relation to 
social life and personality adjustment A review is given of some 
problems involved in the construction of such instiiimenls and of 
circumstances when they can most advantageously be used, 
Calvin Taylor 

I A I 

MEASUREMENT NEWSi^ 

A Personnel Research Section has lecently been established 
in the War Department ujidei the Adjutant General The func- 
tion of the section is to devise and assemble procedmes foi the 
classification of military personnel Di, W. V Bingham is the 
director of this section Among the professional members of the 
staff are Dr T W Han ell, on leave fiom the Umveisity of Illi- 
nois, Mr W M Shanner, on leave fiom the University of Chi- 
cago, and Di Willis Schaefei, formerly of the University of 
Chicago This section has the technical advice of a National Re- 
search Council committee on tho Classification of Military Per- 
sonnel Members of the committee are 

Drs Walter V Bingham, Carl C Brigham, Princeton Univer- 
sity, Henry E Garrett, Columbia Univeisity, L J O’Rourke, 
United States Civil Service Commission, Marion W Richardson, 
United States Civil Service Commission, Can oil L, Shartle, So- 
cial Security Board, and L L Thurslone, Umveisity of Chicago 


As a part of the national defense program the Occupational 
Analysis Section of the United States Bureau of Employment 
Security is making job analyses of occupations in the United 

* Notes for this depailment should be sent to Di MW Richardson, 
United States Civil Service Commission, Washington, D C 
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States Army Over seven thousand analyses will be made and 
]ob specifications will be prepared to aid the Aimy ui making Us 
assignments of personnel The Army is also using the Oial Trade 
Questions which have been developed by the Employment Serv- 
ice, as well as the recently published "Dictionary of Occupa- 
tional Titles " New aptitude tests developed by the Occupational 
Analysis section are being released to both the Army and the 
Navy 

Another field of activities of the Occupational Analysis Sec- 
tion IS that of assisting local employment offices to select rapid 
learners for defense jobs New aptitude tests are being developed 
for this task New trade tests are also being constiucted for 
defense jobs requiring highly skilled workers The greatest at- 
tention is being given to those jobs which arc important both to 
the armed forces and to the civilian defense industries 

The Occupational Analysis Section is under the supervision 
of Dr C L Shartle 

The Washington Psychometnc Society was organized Novem- 
ber 13, 1940, with a charter inembeiship of eleven The following 
officers weie elected: M W, Richardson, piesident, N J Van 
Steenberg, secretary, C, R Brolyei, treasvuei It is planned to 
hold meetings once a month 

Machine methods as applied to the field of measurement 
formed the major subject of discussion at an "Educational Re- 
search Forum" held at the Homestead of the International Busi- 
ness Machines Corporation at Endicott, New York, duimg the 
week of August 26 to 31 A limited numbei of tiaiiscnpls of the 
proceedings are available to those interested Reqviests for tran- 
scripts should be sent to Mr, E C Schroedel, Manager, Institu- 
tional Department, International Business Machines Corpoiation, 
590 Madison Avenue, New York, New York 

The papers presented at the Foium are listed below 
CompuUtioji of Statistical Constants' 

"The Value of the Collator in Using Prepunched Cards for 
Obtaining Moments and Product Moments." — Alan D 
Meacham 

'The Computation of Means, Standard Deviations and Corie- 
lations by Use of the Tabulator When the Numbers ate Either 
Positive or Negative"— Jack W Dunlap 

Summary of Problems in Computation of Statistical Con- 
stants " — Paul S Dwyer 
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“The Design of Tabulating Piocedures in Relation to Auto- 
matic Error Control m Statistical Analysis “ — Charles R 
Langmuir 

“Code Numbers and Coding as Aids to Reseaich " — Herbert A 
Toops 

Chssihcatwn and Prediction 

“Four Aspects of Factor Analysis A Problem foi Which Ma- 
chine Proceduies are Needed "—Hairy H Harman 
“Use of Tabulating and Scoring Machines in Facloi Anal- 
ysis ” — Ledyard R Tuclcei 
“Canonicals ” — Irving Lorge 

“A Successive Approximation Solution for Prediction Prob- 
lems Involving a Laige Number of Vaiiables" — John C 
Flanagan 

“Problems of Classification of Personnel in the Aimy''i — 
Truman L Kelley 

“Army Testing Problems,” — T W Harrell 
Test Construction 

“Computing Difficulty Index and Validity Index in Item Anal- 
ysis by IBM Machines” — John M Slalnakei 
“Item Analysis by Test Scoring Machine Graphic Item Coun- 
ter/^ — John C Flanagan 

“Repetitive Scoring of Inteicst and PeisonaliLy Tests in De- 
veloping Item Weights by an Iterative Process". — Robeil T, 
Rock, Jr 

Testing Programs, 

“The Integration of the Test Scoring Machine with Tabulat- 
ing Equipment in a System of Piogess Tests and Compie- 
heiisive Exainmations,” — J V McQuitty 
“Applications of Electric Accounting Machines m Reporting 
Individual and Group Results in a Testing Program ” — 
Charles R Langmuir 

“The Facilitation of the Analysis and Distribution of College 
Entrance Test Data in a Statewide Testing Program,” — E. L, 
Strombeig 


In the spring of 1939, at the request of a number of school 
teachers and administrators throughout the United States, the 
American Council on Education appointed the National Commit- 
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tee on Teacher Ei^aminations, and author tied it to supervise and 
delegate to the Cooperative Test Service of the American Council 
the task of preparing a battery of objective tests for the exairuna- 
tion of teaching candidates The National Teacher Examinations 
were administered for the first tune in vaiious centers throughout 
the United States on March 29-30, 1940 

New editions of the Teachei Examinations ai e being prepared 
for administration in 1941. The tests cover such aieas as under- 
standing and use of the English language; reasoning ability; 
knowledge of contempoiary affaiis, general cultural information, 
understanding of professional educational points of view, goals, 
attitudes, and methods, and mastery of subject matter to be 
taught All examinations are objective, consisting of '3hort 
answer items involving multiple choice response. In 1941 the 
National Teacher Examinations will be administered two full 
days Approximately twelve hours of testing time are required 
for the examinations 

The dates which have been named by the National Committee 
for the administration of the Teacher E'^aminations m 1941 arc 
March 14 and IS, 

The exammationa have, of necessity, been limited to intellec- 
tual, academic, and cultural materials Other important factois 
that determine teaching success, such aa tiaining, experience, 
personality characteristics, social adaptability, and others are 
judged independently by the local authority to whom the candi- 
date applies 


A revised series of Coopeiative General Achievement Tests 
were introduced this fall by the Cooperative Test Service. The 
revised senes beginning with Form QR includes Test I; A Test 
of General Proficiency in the Field of Social Studies, Test II 
A Test of General Proficiency in the Field of Natural Sciences, 
and Test HI' A Test of General Proficiency in the Field of 
Mathematics These general proficiency tests aie not composed 
of questions dealing with topical content of the fields Covered, 
Instead, each test is divided into two parts; the first, testing foi 
knowledge of the terms and concepts essential to an understand- 
ing of the area in question, the second, testing the student’s abil- 
ity to comprehend and interpret typical materials m the fields, 
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lIIttMA G IIIURSI'ONL 
Chicngo 'Iciicliirs College 

F or many yeais psychologists have been accustometi to 
the pioblems of special abilities and disabilities, These 
aie, 111 fact, the puiicipal concern of the school psychologists 
who deal with childien who cannot lead, have a blind spot foi 
juimbeis, oi do one thing lemaikably well and otiici things 
pooily Tt seems strange with all this expcueiice in diffciential 
psychology that we have climg so long to the piacticc of 
summauziiig a child’s mental endowment by a single index, 
such as the mental age, the intelligence quotient, the peitcntilc 
lank In gcneial intelligence, and othei single average mcas- 
uies An avciage index of mental endowment should be use- 
ful for many educational purposes, but it should not be re- 
garded as moie than the aveiage of seveiai tests Two chil- 
dien with the same mental age can be entiicly dllfcient per- 
sons, as IS well known Thcie is nothing wiong about using 
a mental age oi an intelligence quotient if it is undchtood 
as an avciage of seveial tests, The ciioi that is fieqiiently 
made is iiiteipietiiig it as mcasiuing some basic functional 
unity when it is known to be nothing moie than a composite 
of many functional unities 

The lescarchcs on the piimaiy mental abilities which have 
been in progress foi several yeais have had as then liist 
purposes the identilication and definition of the independent 
factois of mind As the natuie of the abilities became moie 

^Thc shidies lepoUcd in this pnpcL lin\c been cniiicd uiil uiulci the joint 
sponsorsliip of the Chicago Public School^ llic Uiuveiaily of Clucngo, and the 
Amcnenn Council on education 
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cleaily indicated by successive studies, a second pin pose of a 
more piactical natuie has been involved in some of the studies 
This purpose has been to pi'epaie a set of tests of psycho- 
logical significance and practicable adaptability to the school 
testing and guidance progiam The senes of studies will be 
summarized m this papej, the batteiy of tests soon to be 
available will be desciibed, and some of the piobicms now 
being investigated will be discussed bi icily 

Pmviofts Studies 

The first study in this senes involved the use of 56 psy- 
chological examinations that weie given to a gioup of about 
250 college students That study levealed a luimbci of pii- 
maiy abilities, some of which weie cleaily defined by the 
configuiatioii of test vccCois while otheis wcie indicated by 
the configuration but less cleaily defined All of these factois 
have been studied in subsequent test batteues in which each 
pnmaiy factoi has been lepiescnted by new tests specially 
designed to feature the primaiy factois in the piiicst possible 
foim The object has been to constmet tests in which thcie 
is a heavy satin ation of a primary factoi and in which other 
factois aie minimized This is the puiification of tests by 
reducing then complexity 

These latter studies of the separate abilities weie in each 
case made in the Chicago high schools — one study emphasizing 
the perceptual factoi at the Lane Technical High School, 
one study of the inductive factor at the Hyde Paik High 
School) an intensive study of the memoiy factoi or f%ictovs 
in four high schools, and a study of niimeiical ability by 
Coombs in six high schools In each sciies of tests, one factoi 
was represented by a laigc numbci of tests, but all factois 
were well lepiesented. In all of these studies the sdine pii* 
mary abilities weie identified as had been found in the expeii- 
ment with college students, These studies led to the publicii- 
tion by the Anieucan Council on Education of an experimental 
batteiy of tests foL the piimary mental abilities, adaptable 
for use with students of high school oi college age, 
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The identi/icatioii of the s.ime piiniaiy mental abilities 
among high school students as we had ptcviously foiiiicl among 
college students encouiagcd us to look foi diflcieiuiiition 
among the abilities of younger childicii In the Chicago Pub- 
lic Schools, gioup mental tests aie made of all IB, 4B, and 
8B childien in the elemental y schools and of lOB students 
in the high schools The demand foi ,i sciics of tests to be 
used in the gnidancc piogiam foi high school ciUiaiUs and 
the advisability of not making too bioad a leap in age led 
us to select an eighth-giadc population foi tbe next study 

The Eigh}h-G>ade E\pe)imcui 

In view of the pm pose of investigating wliethei oi not 
primaiy mental abilities could be isolated foi chiklicn at the 
fourteen-yeai age level, the construction of the tests consisted 
essentially in the adaptation for the youiigei childien of tests 
previously used with high school students Tn some of the 
tests little 01 no alteiatfon was nccessaiy, while foi othci 
tests it was consideied advisable to levisc vocabulaiy and 
other aspects of the tests to suit the youngei age level /\ 
niimbei of nesv tests weic added to those selected fioin pie- 
vioiis expel iniental batteiics, Sixty tests constituted the linal 
batteiy 

When the tests had been designed and punted, they weic 
given in a tiial form to childien In giadcs 7A and 8A m 
seveial schools Groups of fiom 50 to 100 children in these 
two glades wcie used foi the puipose of standaidi/Jiig pio- 
cediires and, especially, for setting time limits 

Fifteen Chicago elemental y schools weie selected by Tvliss 
Minnie L, Fallon, Assistant Super intenclcnt in chaige of ele- 
mentary education, and by Di Giacc E Munson, Dircctoi 
of the Bureau of Child Study, as expcumental schools foi tliis 
study The tests in the mam investigation weie admiiiistcied 
in the schools by the adjustment tcacheis These adjustment 
teachers had had special Liaining m testing piocechnes with 
the Buieau of Child Study and also had had considerable 
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expel lencc iti giving psy(.hologic.il and educational tests Spe- 
cial instructions in the piocedines foi these tests weic given 
to the adjustment teacheis, as well ns wiittcn instructions foi 
each day’s testing progum 

Eleven himdied and fifty-four cluldien paiticipated in this 
study The complete battciy of 60 tests was given m \ \ one- 
hour sessions to the chiidien in the 8B giacles in each of the 
15 schools The chdtlien enjoyed the tests and, with vciy few 
exceptions, the sustained mtciest and cffoit weic quite evident 
One thing which a psychologist might fcai in such a long 
series of tests would be (luctuatmg movitation on the pait of 
the students Although the adjustment teacheis admiiiisteicd 
the tests, eveiy session was obseived by n membci of oiii 
staff, and we weie highly giatificd by the sustained mtciest and 
effort of the pupils 

In addition to the 60 tests wc used tliice nioic vaiiablcs 
chi onological age, mental age, and sex. The laLtci test data 
weic available in school iccoids They wcic detci mined by 
the Kiihlmaiin-Andeison tests which had been given picviously 
to the same childicn Thciefmc, the battery to be analyzed 
factoiiaUy contained 63 vaiiables 

The total population m this study consisted of 1,154 
eighth-grade children When all the iccofds had been as- 
sembled, It was found that 710 of these subjects had complete 
recoids foi all of the 63 variables, Wc decided to base oui 
corielations on this population of complete recoids lathei 
than to use the laigc population with vaiying nnmbcis of cases 
foi the coi relation coellicicnts Koi convenience of hsirulliiig 
With the tabulating-mdchine methods, the law scoics wcic 
tiansimitcd into single digit scoies fioni which the Pcaison 
product-moment correlation coeflicients wcic computed With 
63 variables theic weie 1,935 Pearson correlation coefficients. 

This table of intci correlations was factored to 10 factois 
by the centroid method on the tabulating machines by means 
of punched caids Successive lotatioiis mnde by the method 
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of extended vectors yielded an oblique factorial matn\ which 
IS a simple stiiicture 

Inspection of the lotated factor lal matiix showed seven 
of the factors previously indicated Memory, Induction, Vei- 
baJ Comprehension, Word Fluency, Number, Space, Percep- 
tual Speed, and three less easily identiriablc factors, One 
of these is aiiothei Verbal f.rctoi , one is Involved in ability 
to solve pencil mazes, and one is present iir the tliiee dot- 
coimtjng tests which were used 

We have computed the inteicoir elations between the 10 
primary factors Oiii main interest centejs on the seven 
primary factors that can be given inter pi etation and, espe- 
cially, on the fust six of these factors for which the jiiteipie- 
tation is Lather more definite Among the high correlations 
we note that the Niimbei factoi is coi related with the two 
Verbal fuctois The Word Fluency factoi has high con elation 
with the Veibal Compi ehension factor and with Induction 
The Rote Mcmoiy factoi seems to he independent of the other 
factors These correlations arc highei than the coiieJations 
between primary factoi s for adults, 

Because of the psychological interest in the coiiclntions 
of the pi Hilary mental abilities, we have made a separate 
analysis of the coiielatioirs for those factoi s which seem to 
have reasonably certain inteipietation If these six pi unary 
mental abilities aic correlated because of some general intel- 
lective factor, then the rank of the correlation matiix should 
be one, Upon examination, this actually proves to be the case 
A single factor accounts foi most of the con elations between 
the primary factors 

The single factoi loadings show that the inductive factoi 
has the highest loading and the Rote Memoj'y factoi the lowest 
loading on the common gencial factor in the priniaiy abilities 
This gcneial factoi is what we have called a second-ordei 
general factoi It makes its appearance not as a separate fac- 
toi, but as a factoi inherent in the primaries and then coi leln- 
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tions ir fiiitliei studies of the pumaiy mental abilities of 
children should levcal this geiieial fnctoi, it may sustain 
Speairnaii’s contention that theie exists a general intellective 
factoi Instead of clepeiKling on the nveiages oi centioids 
of aibitiaiy test battciies foi its detei miiintion, the piesent 
method should enable us to Identify it iiniqiicly 

We have not been able to find in these data a gcneial 
factor that is distinct fiom the piimniy factors, but the sec^ 
ond'Ordei geneial factoi should be of as ninth psychological 
Intel est as the moic ficqirciUly postulated, independent gen- 
eral factoi of Speaiman It would be oiii judgment that 
the second-01 dci gencial factoi found heie is piobably the 
general factoi which Speainmn has so long defended, but we 
cannot say whethei he would accept the piesent findings as 
sustaining his contentions about the geneial factoi Wc have 
not found any occasion to debate the existence of a general 
intellective factoi The lactoiial inetliods wc hive been using 
aie adequate foi hiuling such a factoi, either as a factoi indc 
pendent of the piimaiies oi as a factoi opciating through 
coivelated pnmaiies We have icported on piimaiy mental 
abilities in adults, which seem to show only low positive co\- 
relations except for the two veibul factoi s In the present 
study we have found highei coiielations among the piiniary 
factois foi eighth-giade chlldicn, It is now an intciestiiig 
question to deteimine whether the coiielations among pnmary 
abilities of still yoiingei cliildicn will icvea], peihiips even 
more strongly, a secoiid-oidci gencial factor 

lutet pi einiion of Factois 

The analysis of this batteiy of 60 tests revealed essentially 
the same set of primal y factors which had been found in 
previous factorial studies Six of the factors seemed to have 
Sufficient stability for the scveial age levels that have been 
investigated to justify an extension of the tests foi these fac- 
tors into piacticai test woik in the ichools In making this 
extension we have been obliged to considci caic/ully the dif- 
ference between research on the natuie of the piimaiy fac- 
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tois and the constiuction of tests foi piactical use Several 
of the piimaiy factois aie not yet siiflicicntly clcai ns regards 
pbychological intei pietatioii to justify an attempt to appraise 
them gcneially among school childien The piimaiy factois 
that do seem to be cleai enough foi such pm poses aie the 
following Veibal Compichension V, Woid Fluency W, 
Niimbei N, Space S, Role Mcmoiy M, and riuhiction oi Rea- 
soning R The factois which m scveial studies aic not yet 
sufficiently cleai foi gencial application aic the Pciceptual 
factoi P and the Deductive factoi D 

The Veibal factoi V is found in tests involving veihal 
compiehcnsion, foi example, tests of vocabiilaiy, opposites 
and synonyms, completion tests, and various leading compic- 
hensjon tests 

The Word Fluency lactoi W is involved wliencvei the 
subject Is asked lo tliink of isolated woiils at a lapicl intc 
It IS foi this leason that we have called the factoi a Woid 
Fluency factoi It can be expected m such tests as aiiagiams, 
ihyming) and pioducmg woids with a given initial Ictlcj, 
piehx, or suffix 

The Space factoi S is involved m any task in which the 
subject manipulates an object imaginally in two oi thicc dimen- 
sions The ability is Involved m many mechanical tasks and 
in the nndei standing of mechanical cliawings Such material 
cannot be used conveniently in testing situations, so wc have 
used a latge numbci of tasks which aie psychologically similar, 
such as Flags, Caids, and Figuies 

The Niimbei factoi N is involved in the ability to do 
nuineiical calculations rapidly and accuiately It is not depen- 
dent upon the reasoning factois in problem-solving, but seems 
to be lestiicted to the simplei pioccsses, such as addition and 
multiplication 

A Memory factor M lias been cleai ly picsent in all test 
batteries Ihe tests foi memory which arc now being used 
depend upon the ability to memoii7e quickly It is quite 
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possible that the Memoiy factoi will be biokeii tiown into 
moie specific factors 

The Kcasoiiing factoi R is involved in tasks that rcqiiiic 
the subject to discovei a iiile or piinciplc coveimg the mate- 
iial of the test The Lettei Senes and Lettei Giouping 
tests aic good examples of the task In all these cxpeiimciital 
studies two scpaiate Reasoning factoi s have been indicated 
They ate peiiiaps Induction and Deduction, but we have not 
succeeded m constiucting pure tests of eithei factoi The tests 
which we aie now using aie moie heavily satin a ted with the 
Inductive factoi, but foi the picsent we aic simply calling 
the ability R, Reasoning 

III piesenting foi gencial use a dlflciential psychological 
examination which appiaises the mental endowment of chil- 
dien, It should not be assumed that tbeic is anything final 
about SIX piimaiy factois No one knows liow many piimaiy 
mental abilities tbcie may be U is hoped that futuic facto; lal 
studies will leveal many other impoitant piimniy abilities so 
that the mental pio/ilcs of students may cvenUially be acle 
quate toi appiaislng educational and vocational potentialities 
In such a piogiam the picsent studies aie only a staiting 
point in substituting foi the desciiption of mental endowment 
by a single intelligence index the desciiption of mental endow- 
ment by a piofile of fimdamcntal tiaits 

The Final Tesl Batlety 

In adapting the tests foi piacticnl use in the schools foi 
the appraisal of six piimaiy mental abilities, we must recog 
nize that the new test piogram has foi its object the pi educ- 
tion of a profile for each child, as distinguished fiom tlie 
description of a child’s mental endowment in teims of a single 
mtelligence index For many educational purposes it is still 
of value to appiaise a child’s mental endowment loughly by 
a single measure, but the composite natiiie of such single 
indices must be recognized 
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The ffictoiial matrix of the battciy of sixty tests was 
inspected to find the three best tests foj each of seven piimaiy 
factors In making the selection of tests foi each pumaiy fne- 
toi we consideied not only the factoiial satin ations of the 
testSj which aic, of covuac, the most impoitant consulciation, 
but also the availability of paiallcl foinis which may he needed 
jn case the tests should come into gcncial use Ease of ad- 
min is ti a tl on and ease in iiiidci standing of Llie iiisti'iictions aic 
also jmpoitant coiisideiatjoiis 

The thiee tests foi each piimaiy factni wcic punted in a 
sepaiate booklet and the mateiial was so aiianged that the 
thiee tests foi any fnctoi could be given easily within a 40- 
mimite school peiiocl The mam pin pose of the laigei test 
batteiy was to deteunme whcthci oi not the piimaiy factois 
could be found foi cighth-giade chddicn, but the pin pose ol 
the piesent slioitci battery was to pioducc a piactical, useful 
test batteiy and to check its factoiial composition The se- 
lected tests weic edited and icvised so that they could he used 
toi cithei hand-scoiing oi machine-scoung, The Woul I'lu- 
cncy tests constitute an exception in that none of the tests 
now known to be satuiated with this factoi seems to be suit- 
able foi machme-scoiing 

In Older to check the factoiial analysis at the piesent age 
level, we aiianged to give the selected list of 21 tests to a 
second population of eighth-giade childicn The icsulting 
data weie factoi cd independently of the larger battciy of 
tests Theie weie 437 subjects in this population who took 
all of the 21 tests This population was used foi a new factoi 
analysis The lesults of this analysis cleaily confiimcd the 
picvious study The simple stiiictuie in the picsent ballciy 
IS shaip, with only one piimaiy factoi conspicuously picscnt 
in each test, so that the stiiictuie could be dctci mined by 
inspection foi clusters 

A bfitteiy of 17 tests has been assembled into a sciics of 
test booklets foi use m the Chicago schools An expcilmental 
edition of 25,000 copies has been punted, and the plan for 
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sccuLirtg iioims Oil these tests [iicludes then .'idmuiistiatioii to 
1,000 chiklicn at each half-ycni giadc level fiom ginde 5B 
through the senio! yeai in high school These rccoids have 
been obtained duimg the school ycat 1940 to 1941, The use 
of such a wide age range m standiiidi/ing the test is at fust 
thought, perhaps, lathei stiange- The clToit was made in 
ordci to secure age norms throughout the entiie range of 
abilities found among cighth-gi adc childicii since the tests 
are to become a pait of the testing pioceduic foi all 8B 
children in the Chicago schools Sepmate age nouns will be 
derived for each of the six piimaiy abilities If a single index 
of a student’s mental ability is desired, it is i ccommciided 
that the average of his six ability scoies be used 

As soon as the nouns aic established, the tests will be 
published by the Amencaii Council on liidvic.ition undci the 
title “Chicago Pi unary Mental Abilities Tests “ It is expected 
that the tests will be icady fen distiibution duiing the sum- 
mer of 1941 The noims provided with the tests will be of 
a wide enough lange to make the tests useful at the high 
school and upper giadc levels 

The complete test piogiam consists ol 17 itbta, all ot 
winch have been i educed to niachnic-scouiig loiin except the 
thicc tests for tbc Woid Mucncy factor W In the natuie 
of the ease there seem to be difiicultics in i educing this test to 
machine-scoring form, and hence it has been ictnincd in hand- 
scormg form It should be said, however, that the W tests 
can be scored almost as fast, if not as fast, as the tests winch 
are machine-scored Since all of the tests can be hand-scoieci, 
their use is not limited to schools laige enough to avail 
themselves of the scoring machine The hand-scoilng of all 
the tests IS very easily accomplished by the use of peiforatcd 
stencils to be provided with the tests I-Iaiid-scoring is facili 
tated by the the use of the scoung boaid disliibiited by the 
Stoeltmg Company, 

The new batteiy icpiesents six pumaiy mental abilities, 
namely, Verbal Compiehcnsioii V, Space S, Nunibei N, Mem- 
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01 y M, Wold Fluency W, and Reasoning R They enable the 
skilled psychologist to tabulate a piofile of Si\ Iincaily inde- 
pendent scoies instead of a single mcasiiie, such as the 
intelligence quotient. 

Piincjpals, tcachcis, adjustment teacheis, and school 
psychologists have expicsscd their satisfaction with the piolilc 
of abilities plotted foi each child Piobably the childien 
themselves have found the piofiles most inteiesting and have 
pio/ited most fiom an examination of theii own piofiles In 
the school yeai 1941-1942, these tests will be installed as a 
pait of the educational guidance progiam m the Chicago 
schools by administering them icgiilarly to 8B elemental y 
school pupils and lOB high school pupils 

Some of the featuics of the tests should be mentioned 
The tests arc so lUiaiiged that machine-scoiiiig and hand-sco]- 
ing tests aie ducctly comparable and will have the same 
norms The child's task does not vary with the type of scoi- 
mg, only the scoici’s job is changed. Anothci fcatiiic is the 
use of forc-e\cicisc booklets punted on yellow papci The 
time limits foi the piacticc cxcicises aic api)io\imaLc WJicn 
a test pioper is staited, the student places his white test book- 
let on top of his yellow piacticc booklet, and the cxammcis 
and pLoctois can check at a glance that cveiy child is woik- 
mg m the right place The tests piopei aic to be timed 
exactly The thicc tests of each of the six abilities aic 
ananged in a booklet foi admiiiistiation within a 40-niinute 
school peiiod It is iccommended that the successive booklets 
be given on successive school days 

Fiuihci PiobfcDis 

One of oLir piincipal leseaich uitcicsts at the piescnt 
time is to dcteimiiie whether pumaiy abilities c.in be identi- 
fied in children of kindeigaiten oi fiist-giade age A seiics 
of about 50 tests is well under way, and some of them aic 
now being tried with young children Tf wc succeed in isolat- 
ing primal y abilities among these young childien, oia next 
step will be to piepaic a piactical batteiy of tests for that 
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age A subsequent pioblem will be to make cxiDeiunental 
studies of papei-and-pcndl tests foi nppi .using the primaiy 
abilities of chiklicn in the intci mediate giadcs, approximately 
at the foiiith-giadc level We aic fan ly confident that such 
tests can be piepaicd foi use in the intciniediate giades 
It IS a long way in the futuic, but it is Intel csting to spec- 
ulate on the possibility of using the tests of the pi unary 
mental .abilities as the tool with ivhich to study fundamental 
psychological pioblcms of mental giowth and mental inheut- 
ance Absolute scialing of the tests at the difieient age levels 
will make possible studies on the latcs of ticvelopment of the 
sepaiate abilities at various age levels Modifiability of the 
abilities will be aiiothei pioblem to which we shall latei 
turn attention 
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I' 0 WItllAMSON AND L S IIORDIN 
Universiiy of MiiincKoln 

S ystematic eHoits at evaluation are a lelaiively lecent 
development m the field of counseling The foim of 
appiaisal has tanged fiom “veibal leseaicli'* to simple statis- 
tical analysis Not all of these studies have avoidctl the pit- 
falls, of winch theie aie many, to be foiuul in this undeilak- 
ing The assumptions, methods, anti weaknesses Involved in 
the vaiious evaluation appioaclics aic summiiii/ed in a pie- 
vioiis papei (10) 

The piesent papei, one of a luimbci to be icpoiLed, sum- 
maiizes an expeiimeiit designed to evaluate a ceitaiii type of 
counseling Since oui conclusions aic applicable only to coun- 
seling based upon the philosophy and piocedincs employed 
at the Testing Biueau of the Univeisity of Minnesota, thi'^ 
type of clinical counseling should be desetibed 

This clinical counseling has as its purpose assisting the 
student to choose and make piogress towaid educational and 
vocational objectives which will yield maximum satisfaction 
It IS assumed that this end can be accomplished best by aiding 
him to set his aspuations in teims of the level of his poten- 
tialities Natiually his potentialities must fust be analyzed 
befoie a diagnosis of any discrepancy between aspiration and 
ability can be made and befoie assistance can be foithcoming 
fiom the coLiiiseloi 

^Assistance in the preparation of this mnlfliial was furnished by the per 
sonnel of Work Projects Admiiiislinlion, Ofhcinl Piojeci No 65-1'71-HO, Siib- 
ProjeCtNo 93 
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The case data upon which analyses and diagnoses aie made 
consist of standardised ability tests, peisoiiality and mteiest 
inventories, questionnaire lecords, and non-qiiantified informa 
tioji collected fiom the student, Ins associates, and his paients 
This accumulation of infonnation must be intcipietcd on the 
basis of an integiated pictiiic of the individual piovidcd 
thioiigh personal mtei views In othei words the connscloi 
deals with a unique individual lathei than with a gencialwcd 
conception of a group of individuals 

The inteiview piovides the medium tluough winch coun- 
seling IS peisonaliTcd and tlnough which the student is assisted 
in making his decisions While the decisions that the student 
accepts are and should be Ins own, the counselor sometimes 
plays a peisiiasive lole in that he oigani 7 ,es relevant case data 
to highlight the alternative coinses of action fiom winch the 
student chooses Once the student has made the choice, the 
counscloi has the task of aiding him to orient himself to his 
iiiteicsts, attitudes, and abilities, and Ins cnviionmcnt, home 
and family, lecication, and education foi the most successful 
achievement of the chosen objective 

A fullei descnption of tins clinical counseling process has 
been presented clsewheie (9, li) Only by means of an 
acciuate conception of what the counselor is doing and what 
he IS trying to do can any evaluation of that counseling be 
meaningful Moreover, wc should not attempt to generalize 
our conclusions to include any othei type of counseling than 
the one studied 

In attempting to evaluate this climeal counseling, we be- 
lieve that a criterion flexible enough to avoid artificial fiag- 
mentation of the individual provides the most adequate design 
for experimentation Essentially such a design involves a 
judgmental compaiison of the individual’s adjustment status 
before and after counseling. This method — essentially the 
non-statistlcal weighting of variables to form a composite 
estimate — was used in a previously reported study (11 . chap 
IX) In this study an estimate of the degree of the student's 
cooperation was used as a means of control The control 
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lies m the compaiison of those students who did with those 
who did not follow the counseloi's iccoinmendations 

The pioccss of making these evaluative judgments involves 
thiee phases; (a) the pielimmaiy icview or analysis of 
the ease data, (b) the follow-up intcLvlcw, (c) the case 
evaluation 

In the {iLSt phase of the evpenment, all student cases wcic 
Independently and ciitically lead by two tiaincd woikcia wliose 
hinctJons weie to analyse and iccoid all infoiniatmn con- 
tained 111 the case foldei Any disci ep.incics between the anal- 
yses of the two leadcis were leconcilcd or adjusted in con- 
ference with the staff memheis concerned with the pioject. 
The case levieweis also compiled questions conceiiiliig the 
piesent status of the student, his adjustment to lus oiiginal 
piobleni oi pioblems, his adjustment to the counsel given, and 
any othei peitinent iiifoiiTiiition, These questions wcic used 
subsequently in the follow-up intcivicw. 

For the second step all student cases wcic called in foi 
a follow-up mteiview Cases which weie incomplete because 
of msufhcient intciview contacts oi incomplete test battery 
and which could not be i cached for follow-up mtei viewing 
on the campus weic i cached cithci thiough a qucstionnaiic 
01 an mteiview in the home I'oi those students who had 
left the University, infoimation was collected, and used, con- 
cerning then adjustment to then jobs and their satisfaction 
with that out-of-school adjustment The follow-up Intciview 
yielded infoimation conceinuig the extent of success oi failiiie 
achieved by the student m solving each of his ouginal piob- 
lems and the extent to which the counseloi’s advice had been 
followed subsequent to the oiiginal counseling interviews. The 
student’s own statement of the degree of his satisfaction with 
bis solution of the problems and with University Testing Bu- 
leau counseling and lecommendations or any other intcipre- 
tatioiis or evaluations that he made were specifically recoiclecl 
The interviewei did not interpret or evaluate this m forma- 
tion The purpose of the follow-up interview was, essentially, 
to obtain the factual data on the present status of the case 
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Tiamcd evaluatois iiexL ciitically icvicwcd the oiigmal 
case data and the follow-up interview icpoit to auive at a 
judgment of the extent to Avhicii the student had adjusted the 
pioblems foi which he had oiigmally sought counseling The 
effectiveness of the counseling was evaluated in teims of the 
following counseling functions ov sei vices, 

1, Diagnosis of the student's vocal lonal and educational 
possibilities 

2 Advice in making appiopiiatc choice of a vocational 
field and in sccuiing the i elated educational training sequences, 

3. Counseling as to lecognition of, and alleviation of, 
distuibing factois fcmotional, cducaLional, economic, health) 
which may mteifeic with oi pievent the acceptance of pioper 
vocational choice and the achievement of appiopiiatc ti'aniing 

4 Assistance in the discovciy and iitili/ation of pcisonal 
lesouices in effecting an adjustment 

5 Guidance iii the use of all Uiuveisily peisonncl le- 
sources In diagnosis and counseling 

The student's adjustment with legaicl to vocational choice 
and his piogiess towaid achieving satisfactory tiainliig foi 
that choice wcie judged by means of the following cutcu.i 

1 Choices made in line with aptitudes, inLcicsts, work 
habits, personality, etc 

2 Piogiam of studies in line with these choices and the 
student’s qualifications. 

3 The student’s satisfaction with vocational choice. 

4 Pi ogress in achieving tiaining for objectives in teinis 
of the capacity of the student to piofit fiom such tiamiiig 

5 Alleviation of factois which inteifeied with the mak- 
ing of a satisfactory vocational choice and with acquuing 
the necessaiy tiaining, e g, paiental dominance of choice, 
inadequate study skills, etc 

In making then appialsals the evaluatois studied the stu- 
dent's mteiests and aptitudes, the counselor's interview 
notes, the student's lepoited comments., and his giadc recoid 
achieved hefoi e and aftei counseling All of the information 
was weighed and balanced with lefcience to the five criteria 
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befoie a judgment was made of the dcgiee of adjustineiit 
achieved by the student subsequent to counseling The degiee 
of coopeiatioii was independently judged in the same maniiei 

The following five categoues’* foimed the scale of 
adjustment 

S^lisfactoiy Adpishncnl — I The student is satisfied with 
his vocational adjustment at the time of the lollow up lUtei- 
view In some cases the student’s dissatisfaction will not be 
a deteirent to a lating of satisfactoiy adjustment Tn instances 
wheie the student’s aspiiations aie fat above liis level of abil 
ity, he IS consideied satisfactouly adjusted if be accepts the 
fact that his ambitions must be pitched at a lowei level 

2 In the iiitei viewer’s judgment the vocational choice 
and adjustment of the student aic adequate, based upon 
aptitudes, inteiests, and subjective factois levealcd thiough 
iiiteivicws 

3 Theie has been an alleviation of disti acting lacLois 
which Intel feie with vocational choice and piofcssioiial ttiiiii- 
ing such as inadequate sociali/.ation, mental conflicts, financial 
pioblems, health handicaps, and any othci pioblcms 

4 Achievement in a given tiaming pi ogiam is commensLi- 
late with aptitudes and intciests 

Some Pfogiess Towaid Adjust moil - — ^Thc siuclciit has 
not yet leached a satisfactory adjustinenl, accoiding to the 
pievioiisly stated ciiteiia, but is evidently staitcd on the load 
and may eventually leach the desired objective He may have 
come to the counseloi with a numbei of pioblems involving 
vocational choice, classification in college classes, and social 
adjustment and peisonal pcculiaiitics In the follow-up nitei- 
view it may be found, foi example, that he has succeeded in 
settling his vocational question but that he is still stiiiggling 
foi mastciy in icgaid to social adjustments 

No Change — ^This classification is used for those cases in 
which the pioblems remain the same as at the time of counsel 
ing While the passage of time will usually make a pioblem 
more seiious, the designation of ’^slightly worse” was not ap- 

^Described and illiislrnlcd in an cnrlici study Sen rcfciciiLc U, chap IX 
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plied unless u choice point had actually been passed Thus 
a sophomore who has not yet made a vocational choice would 
be classed as unchanged, but a junioi without a vocational 
choice would be in a moie seiious position and theiefoie \you1c1 
be classified as slightly woisc Juniors should have begun 
specialization if they aic to make “noimal” piogiess towaid 
graduation 

Shghtly JV oue — I'his Is a condition in which the solution 
of the oiigiiial problems seems slightly more i emote and the 
factors which existed at the time of the first counseling contact 
still exist and aie accentuated 

Much Worse — Those cases wheic the student’s piobleins 
aie more seveie and the solution much nioie remote oi less 
piobablc of achievement 

The judgments of the degicc of coopeiatioii weic based 
upon the following categoiies . 

Followed advice wholly — The student followed the couii' 
sclor's advice with lespcct to the most dominant oi impoitant 
oiiginal problems 

Followed advice in jyail — The student citbcr paitudly fol- 
lowed the counselor's advice with icspcct to the clucf pioblems 
01 completely followed advice with lespect to some but did 
not follow advice with respect to otheis 

Did 110^ foJloxv advice — The student did not follow the 
counselor's advice in legaid to any of the mam pioblems 

In order to deteimine the reliability of classification of cases 
according to the foiegoing two sets of categoiies, an “outside” 
judge was called m to make independent judgments of the 
adjustment of a landoni sample of 247 cases A cocflicient 
of correlation of 82 was found between the “outside” judge's 
classifications and those made by the cvaliiatois This cocfli- 
cientmay be interpreted as a high index of validity oi as a fan 
index of reliability, accoidmg to the reader's own conception 
of the meaning of these two tcims, In ovci half of the cases 
where a disciepancy occuiied, the “outside” judge had made 
a highei classification than had the two oiigiiial judges This 
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would seem to indicate that the evuhiatois had not ovei ^esti- 
mated the eftectiveness of the counseling 

The question aiises as to how much inHuenc-c the student’s 
subsequent academic achievement (available to the cvaki 
ators) had on the judge’s estimate of adjustment and coopeia- 
tion The conelations between lioiioi-point latio achieved 
aftei counseling and judgment of adjustment wcic ,2'1 and 
39 lespectively, foi Gcncial College and the College of Sci- 
ence, Litciatme and the Arts " The dillcience between these 

coefficients, of boideiline significance (Dr/S K =2 0^), 

Ur 

may indicate that academic adjustment is moie closely lelated 
to judgment of total adjustment foi SLA students than foi 
Geneicil College students This does not seem iimcasonablc, 
since SLA students aie gcneially committed to caiccis in which 
academic achievement is one of the most immediate lequisites 
foi success The correlations of honoi -point latio with judg- 
ment of cooperation weic of negligible magnitude 16 ± 05 
foi Gcneial College studejits and 17 it 03 foi SLA students, 

In all, data weic collected on 987 complete student cases 
who used Umvcisity Testing Bmeaii sci vices during the years 
1933-34-35 For the purposes of this study it was deemed 
dcsiiable to analyze as homogeneous a population as possible 
without the saciificc of too much data Foi this leason 498 
students from SLA and 195 students fioni the Gcncial College 
weie selected Classified according to then status at the time 
of counseling, in the SLA gioup were 154 pi e-college cases, 
176 fieshmcn, and 168 sophomoies The Gencial College 
gioup contained 41 pie-college cases, 125 freshmen, and 29 
sophomoies The pi e-college cases weic high school seniois 
01 leceiit giaduates who came to the Biiieaii foi counseling 
ill the gpiing and summei immediately piecccling cniollment 
in the Univeislty, 

That the gioiips chosen foi study weie a fauly satisfactory 
repiesentatioji of the total lange ol ability and achievement 
111 the undeigiaduate classes of these colleges can be dcmoii- 

Uleieafter designated ai SLA 

^Computed accurdin^ to Fisher's method (1 pp 208 10) 
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stiated fiom the distiibutiotis of aptitude test aiul high school 
petcentile scoies Of the SLA students, 62 6 pei cent and 78 I 
per cent fell at oi above the fiftieth percentile in aptitude and 
high school peiccntile scoies, jcspectively, while 82 8 pei cent 
and 65 2 pei cent of the Geneial College students fell at oi 
below the fiftieth peicentile in the same vai tables* Too often 
there is the tendency to assume that only low ability students 
have a desire foi or need of counseling The distiibutions of 
the SLA gioup would seem to lefiite this assumption The 
General College population piovidcs us with the oppoitiinity 
to deteiminc whctliei counseling can be equally effective with 
low ability students 

The lepresentativeiiess of oui gioups in teims of SLA 
and General College freshmen was detcimined by a compau- 
son of high-school aveiage giacles tiansmnted into pcicentiles. 
Uiifoitunately, statistics on sophomoics iii these colleges weie 
not available Because of the known elimination of fieshnicii 
with lowei peicentiles, the soplioinoie population should be 
Inghei on the average Since oui expeiimental population 
consisted of students fioin both classes, this analysis of lepic- 
sentativeness is not precise* The freshmen m our group wcic 
compaicd with icpieseuutive SLA and Geneial College 
samples. For SLA the combined mean foi a sample of 2,157 
fieshmcn students of the fall classes of 1933-34-35 was 65 45 
as compared with 69 59 foi our expeiimental freshman gioup 
Although this small diffcience of 4 14 is icliable (C R of 
3 39), It does not lepiescnt a veiy significant one as fai as 
the purpose of this study is concerned The compaiison of the 
means of the Geneial College gioups yields similai lesiilts. 
The combined mean of representative fieshmen of the 1931 
and 1935 freshman classes was 34 00“, foi oiii gioup it was 
40 25 Although the diffeience is somewhat laigei than that 
in the SLA group, it is not so reliable (C R of 2 64) 

We may conclude from these two analyses of the niituie 
of OUI counseled groups that they weie geneially icpicsenta- 

■^From unpublished data collected by Dr Rulh Eckert of the Univcisily of 
Minuesota 
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tive of then total populations and of the total lange of ability 
to do college work It is inteicsting lo note that the students 
who aic counseled by the Testing Bin can, contiaiy to the 
opinions of many, aie not the students of infenoi ability, but, 
if anything, aie slightly siipeiior to the gciicinl iindcigi .uliiatc 
population of these two colleges. 

Results of ihc E\pcnineul 

Degjec of Coopei aiioii aud /Idjusiuienf — Picvious studies 
of the effectiveness of counseling which used simdai methods 
have leported lesiilts in teims of pcicentagcs The propoi- 
tions of oui gioups who were classified as satisfactorily oi 
partially adjusted (82 8 per cent of SLA and 86 2 pei cent of 
GcJieial College) compaie favoiably with those icpoited 
in English studies Oakley (3) and Maciae (2), woiking 
with small populations of youngci students, lepoited 95 pei 
cent and 55 pci cent, lespectively, as the proportions who 
followed advice and who weic satisfied and successful in then 
occupational adjustment Rodgei (4), with a Iniger popula- 
tion, lepoits 79 pel cent siicces'sful adjusLinciit, Seipp (5), 
using a methodology almost identical with oiiis, analyzed the 
case lecoids of 100 adults diagnosed and advised by the 
Adjustment Seivice of New Yoik, She found that 57 pci cent 
made a satis factoiy adjustment subsequent to coiinseling, Oui 
lesults aic even nioic impiessive when analyzed m tcims of 
those who cooperated in following the counseloi's advice In 
these teims the peicentagc of the SLA students satisfactoiily 
01 paitially adjusted is 93 5 and the percentage of General 
College students is 96 3 

Oiii data also indicate that the counseling was equally effec- 
tive, if not moie so, in gaming the cooperation of the student 
For SLA 70 9 pei cent coopeiated wholly and 20 1 per cent 
paitly, while foi Geneial College the pcicentagcs weie 69 7 
who coopeiated wholly, and 24 1 paitly, Vitcles (7), diag- 
nosing and advising 75 adolescents, found that 58 pei cent 
followed advice completely and 21 per cent paitly. 

Since the SLA and Geneial College groups diffeied so 
maikedly in college aptitude, it was interesting to determine 
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whether theie wfis nny real dlEfeience between the adjustment 
classifications of the two gioups of experimental cases To 
test this hypothesiSj the chi-sqiiaie test of independence 
was vised." The result (chi-squaie value of 3 48, /» >■ 05) 
indicates that theie was no diftcrence in the adjustment 
achieved by the two gioups A similar analysis of the coop^ 
eiatioji classifications yields a similar icsult (chi-sqiiaie value 
of 2 51, /» > 05) A fiuthei analysis involving the length of 
the student’s attendance in the Uiiivcisity was geiieially uniC' 
lated to eithei adjustment oi coopciation The chi>squaie 
values foi the groups weie insignificiint in value 

Ad)ustn]ent venus Debtee of Coopootiofi — In addition to 
these direct analyses, we have attempted to shed light on the 
definition of the conditions which make adjustment moie 
probable as an outcome of clinical counseling Fiist and foie- 
most of these conditions is that the student coopeiate with 
his counseloi Anyone who has had intimate expeuence in 
counseling will have observed that the cultivation of a coopei- 
ative attitude usually piecedes effective counseling That 
the greater piopoition of adjusted students found among 
those students who cooperated is not accidental is cleaily indi- 
cated by the test results of the independence of these two 
vaiiables The chi-squaie values of 115,62 and 47 44 foi 
SLA and Geneial College students aic both highly significant 
(/? < 01) This means that we may assume that a student 
who coopeiates with the counselov in attempting a solution 


'^Thia stntiahc may be used Lo teat tlie independence of distributions from a 
real or hypothetical distribution (6 chap 1] As used in this study, the 
expected distributions were based upon the proportions of the five classes of 
ad|ustmetit observed in the total distribution The formula for compulation 
appropriate for this type of analysis is 

1 


yielding a value which, by use of a table of chi-stiune clistribiilions, is irnnslated 
into an estimate of the probability that such n vnluc could have been obtained 
for additional samples drawn from the same genern] populullon We shall use 
the conventional five per cent and one per cent points ns our confidence limits 
These points are equivatent to values two and three standard deviations from 
the mean. Because of the amall number of eases, some of the categories were 
combined in all of the chi-aqiiare tests used Five was the smallest niimbei of 
cases permitted in any one category, 
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of pioblems will in all piobubility achieve satisf.ictoiy adjust- 
meiit as defined above Only the Gcncial College sophomores 
did not exhibit a statistically significant iclationship. The 
restriction of the lunge of the v-iuablcs ncccssitiUcd by the 
small mimbei of cases in this gioiip may explain this fact 

Expetiancy of /I djustment /hcoidtuff lo Type of Ptoblem 
— A pievioiis study by Williamson (8) has shown that coun- 
selors tend to specinli/c in the types of problems that they 
treat It Is impoitant, theiefoie, to detcnninc the effective- 
ness of counseling with icspcct to diffeicnt types of pioblems 
Since in most cases students expciience moie than one piob- 
lem, classification in any one categoiy, eg, vocational 
pioblem, will include students wlio may also have an educa- 
tional pioblem, a social pioblem, oi any othci pioblcm oi 
combination of pioblems In view of this fact. If the voc.i- 
tional categoiy shows a significantly gieatci piopoition of 
adjusted students than the educational oi the emotional 
categoiy, then evidence of the dilleicntial effectiveness of the 
counseling will have been discoveied 
An analysis of oui data gives a cleai indication that the 
adjustment expectancy is not so maikcd foi social personal- 
emotional pioblems as foi vocational and educational types 
of pioblems While not all of the dilfcicnccs aie statistically 
significant — tho total General College gioiip and the Gen- 
cial College fteshmen showed the significant ones (chi-sqiiaic, 
29 84, /) < 01 and chi-sqiiaie, 32 99, < 01) — the ticiids 

aie consistently in the same diiection Since it has already 
been shown that coopeiatjon can be assumed as a counseling 
condition necessary to adjustment, it is not suipiising to find 
that theie is a gi eater expectancy of coopeiation foi voca- 
tional and educational pioblems than tor social-peisoiial- 
emotional ones 

Evpectciucy of A dptslmcHi Accoi diug to Status of Eoca- 
houal Choice Since the counseling being evaluated in this 
expel iment is piimaiily educational and vocational, the types 
of changes in vocational orientation required should be of 
impoitancc for the expectancy of adjustment Four possibili- 
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ties weie clefmed (a) corifiimatioii of the student's choice 
by the coiiiiscloi , (b) recommendation by the counselor of 
some choice other than the student’s, (c) lecommcndation of 
a choice by the counseloi, because of the student’s indecision 
at the time of the oiiginal contact, (d) defeiment of choice 
on the counscloi’s advice at the time of oiiginal contact It 
had geneially been assumed that the counseloi is more likely 
to biing about adjustnicnt when he has only to confirm the 
student’s pievious choice The icsults of oiii expenment do 
not siippoit this assumption They indicate that it makes no 
diffeience, foi this type of counseling, whethei the student’s 
choice was confiimed oi changed oi whethei he was undecided 
at the time of the hist inteiview But in those cases wheie 
choice is deferied, the expectancy of adjustment is signihcantly 
less (chi-s(]iiare of 2B,59, /» < 01) In the case of coopera 
tion, what "ought to he tme" actually is tiue As one would 
suppose, greatei cooperation is to be expected from those 
students whose vocational choices vveie confiimed (chi'Squaie 
of 157, p< 01) 

Aptitude aud Achievement ni Relation to Adjustment and 
Coopeiaiwu — One might expect that ability and previous 
achievement of students who come foi counseling would be 
positively I elated to expectancy of coopeiation and adjust- 
ment This pioblem was attacked by studying the aptitude 
and achievement chaiacteiistics of students in each of the co- 
opeiation and adjustment categoues The analysis of the 
vaiiance in aptitude test scoies gives evidence that this assump- 
tion cannot be held in teims of the ability test used ’ The 
vauance latioa weie of such a small degree that the piobabili- 
ties that they lepiesented the same population weie gieatei 
than live in a bundled This means that low ability students 
are just as likely to be coopeiativc and adjusted as high 
ability students 

On the othei hand, the analysis gives reliable evidence 
that high school achievement is positively i elated to coopera- 

^Snedecoi’a tables of J*' (6 p 174) wcfe used lo csiimote tlie probabililiea 
□f gettiTig as Ibte® vnriance iptio from aomples ot a homogciieau') population 
Here agiiin the pc\ cent and one per cent pomls ■were iftkeii os ihe Iithus 
of conlidence 
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tioii and adjustment (Gcneial College, F is 8 09, /> < 01, 
SLA, IS 5 45, j) < 01) This 1 elation is fiiithei emphasized 
by the finding that foi any dcgiee of adjustment, those stu- 
dents who coopeiated weic, foi the most pait, those with 
pieviously highei achievement Previous college achievement 
could be analyzed validly only in i elation to coopeiatlon, 
since It alicady had enteicd into the estimate of adjustineiU 
The lesiilts heie aic not conclusive While the SLA data 
yielded a significant variance latio (8 22, p < .01), the Gcn- 
eial College data did not 

Niimb^} of liitcj views veisvs /IdjushneiH (Did Coopcia- 
tion — With lespect to SLA students, vaiiation in nunibci of 
intei views indicates that the counscloi had the most inteivicws 
with students who wcie paitially adjusted (Geneial College, 
7^ is 4 13, /j < 05, SLA, F is 20 84, /» < ,01) Tims those 
students who aie satisfactonly adjusted, chaiacteiisticalJy, 
do not lequiie so many mtei views Likewise, those students 
whose maladjustment is of such a natiiie (e g , vciy low ability) 
as to oftei little piobability of adjustment aie not mtei viewed 
so fiequently It seems, then, that those students who piescnt 
difficult pioblems hut give piomise of adjustment seek counsel- 
ing intei views most fiequently Foi SLA theic is slight 
evidence foi a positive lelatlonship of inimbei of iiitci views 
with judgment of degiee of coopeiation {F is 3 07, /> < 05) 
In the case of Geneial College students, theie is a negative 
lelationship between adjustment and the niimbei of intei views 
The students in the low adjustment categoiics appaiently 
show a gieatci willingness oi aie moic encouiaged to letuin 
for fiiithei counseling than aic the moie satisfactoiy adjust- 
ment gioups 

Time Ijitctval venns Evaliialion — The significance of the 
time elapsed between the hist counseling mtei view and the 
follow-up interview should be of value m indicating the 
optimum period foi an evaluation cxpciiment Since theie is 
no leason to suppose that special selection has opeiated in the 
selection of the time at which the adjustment gioups weic 
studied, it is leasonablc to infci that observed dilTeienccs aic 
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differences m time necessaiy for rcuching thnt level of adjust 
ment While the previous analysis of SLA data indicated a 
greater niimbci of interviews for the students classified in the 
paitially adjusted catcgoiy, it is evident that students in this 
category lequii'cd the shoitcst time to achieve then degree of 
adjustment Analysis of the Gencial College data suppoits this 
icsult, a pciplexing one Howcvei, a more intcipictable lesvilt 
is secured when the data aie analyi^cd in teims of the inteuc^ 
lation of cQQpeiatioii and adjustment The ticnd is m the 
duection of a shoitcr time intcival foi students m any degiee 
of adjustment who coopeiated to a gieatci dcgiec with the 
coungeloi The infeiencc may be made that those students 
who coopeiated i cached a given level of adjustment in a 
shorter time The diffeiencc avciagcs n little ovei two months 
in an aveiage evaluation peiiod of 16 months The P value 
of 3 4 is beyond the one pei cent point 

iSm/i III if I y 

This evaluation of the clinical counsclmg piacticcd in the 
Testing Bureau of the Univeisity of Minnesota has attacked 
two basic pioblcms (a) What piopoitions of students weie 
aided by the Bmeau’s counseling to achieve a bcttci adjust- 
ment? (b) What conditions and chaiactciistics of counseled 
students aic most conducive to a favorable prognosis of sub- 
sequent counseling? 

In answer to the fust qiiestioiij counseling was effective in 
achieving the cooperation of and in improving the adjustment 
of over 80 per cent of the students in our groups This is 
especially significant in that the analysis and classifiLation of 
cases were carefully defined and conti oiled, having been made 
by judges who had not been involved in any of the counseling 

The conditions and characteristics favorable to adjustment 
include the following 

I Cooperation with the counselor was positively related 
to adjustment and those students who cooperated reached then 
level of adjustment in a shortei period of time than those who 
did not 
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2 Students expciienciiig ediiciitional and vocational piob- 
lems were moie successfully counseled than wcic those with 
dominant sociai-peisonai-emotioiial pioblems 

3 Contraiy to belief, oui data indicate no dilleieiiceb in 
adjustment among counseling cases classified as vocational 
choice confiimed, altcied, oi undecided at the fust contact 
But, if vocational choice is dcfciied by the counseloi, the plug- 
iiosis of adjustment is less favoiable 

4 Highei high school or pievious college achievement is 
positively related to cooperation and adjustiiicnt But level 
of ability, as measiued by the aptitude test used in this expeii- 
ment, is not i elated 

These conclusions may be interpieted as liinitatiojis cithci 
of the students involved m of this type of counseling In the 
case of the type of pioblem, it is likely that a limitation of 
counseling is disclosed Counseling that is ecliicationally and 
vocationally oiiented is not likely to deal so cflectivcly with 
social-pcisonal-cmotional pioblems On the othci hand, it 
does not seem probable that any type of counseling oi impiovc- 
iiient in tieatmeiit techniques can do much foi a student with 
a veiy low achievement backgiouiid inaofdi as the types of 
adjustment involved iii this evaluation cxpeiimcnt aie 
concerned. 

Cel tain relations of an ambiguous natiiie and therefore 
demanding fiiithei study wqic observed Theic was evidence 
that the counseloi conducts moie iiitei views with students who 
aie judged as partially adjusted, yet this same gioiip reached 
their level of adjustment within a shoitei pciiod of time 
Oiu data do not indicate whethei oi not the counseloi tends 
to intensify his woik with ccitain students by conducting many 
mtei views within a shoit peiiod of time 

There is one conclusion that this study should linve made 
clear The evaluation of counseling is not a casual piocess, 
easily cauied out Indeed, such a study icpiesents a combina' 
tlon of caieful niul iigoioiis case ic.uliiig, many days and weeks 
of interviewing, prolonged cleiical and statistical laboi, and 
above all a peiiod of patient waiting for the counscliiig cases 
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to matluc to tlic stage whciein adequate data aie available 
foj ciitJcal evaluation 
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CONTRIBUTION OF TESTS TO RESEARCH IN I'llE 
FIELD OF STUDENT PERSONNEL WORK 


RALPH W TYLER 
Univcisily of Clucngo 

T he use of tests IS fundamentiil to many aspects of 
student peisoniiel woik* In the selection of stiulcnts, m 
identifying then potentialities, then pioblerns, and then diffi- 
culties, In checking on the effectiveness of pioccdiiies used 
in providing foi pcibonal development, in vocation.il place- 
ment and follow-up, peisonncl woikcis have learned to use 
a wide lange of tests and to depend on the lesiilts of tests 
as a basic pait of the pcisonncl piogiam Although the place 
of tests in the piactiie of pcisonncl woik has been well out- 
lined, the contiibutlon to be expected fioin tests in comicclion 
with peisonncl lescaich has not been so clcaily Indicated I 
am diffeien tinting leseaich fiom piacticc in the field of student 
peisonnel woik by defining leseaich as the pioccss by which 
basic facts, theoiies, piinciples, instillments, and piocediues 
are developed, thus piovidmg a lational fiainewoik upon 
which the piactice of student peisonnel woik can be undei- 
stood and elaboiated This distinction may peihnps be made 
deal Cl by illustiation 

It is a common piactice of the student peisonnel officci 
to administei leading tests to incoming fieshmen, to study the 
lesiilts, to identify ceitain students who leccived lelatively 
low scoies on the leading tests, ancl to lecommend a icmcdial 
progiam in reading foi some of these students Rcsenich 
which finds out what leading demands riie likely to be made 
by the various fieshmen courses, which devises valid instiii- 
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ments foi measiitiiig these lesidiiig abilities, which estimates 
the probable fiequency of inadequate reading abilities among 
fieshmen, which develops theory and principles regarding the 
1 elation of leadihg development to other aspects of the 
student’s development, and which establishes the probable 
validity of various types of remedial reading piocediues 
would represent the essential framework upon which improved 
personnel practices relating to reading can be built Practice 
and research aic complements in a sound professional growth 

Because student personnel work may be concerned with 
all aspects of the personal and social development of students, 
its problems relate to many pieviously oigam/.ed fields of 
jescaich such as physiology, psychology, sociology, anthro- 
pology, psychiatiy, and education Obviously, personnel work- 
ers have drawn and must diaw upon these vaiious oigani/cci 
fields of leseaich foi many of then concepts, Instiumcnts, and 
piactices However, many pioblems which the student pei- 
sonnei woikei faces cut across two or more of these fields 
and are likely to involve research aspects not adequately 
investigated by any one of these disciplines alone The prob- 
lems which do involve two or moie organized disciplines aie 
the pioblems which in gciieial irmst be attacked by reseaich 
woikeis in the field of student peisomicl May I indicate 
some ot these problems and suggest contubutions which tests 
have made or can make to research on these problems? 

One major leseaich problem is to delineate cleaily dcsii 
able goals for a student personnel program. Accepting the 
general function of personnel work to be the facilitation of 
weU-iouncleci personal and social development of students, it 
IS evident that this function must be defined moie cleaily in 
the case of a given college or type of college so as to indicate 
the aspects of development to be ptomoted and the desired 
1 elation among these various aspects This clearer picture of 
the phases of student development to be given attention and 
then relation is essential to the intelligent direction of a pro- 
gram aimed at facilitating well-iounded development of the 
individual student 
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It IS obvious that a piofession should have its goals clearly 
and definitely in mind; it is not so obvious that the foimula- 
tion of these goals foi the field of student peisonnel is a 
leseaich pioblem of consideiable magnitude The difficulty 
of the task is paitly due to the complexity of human develop- 
ment Wcll-iounded peisonal and social development includes 
physiological, psychological, and social aspects Inn theimoic, 
these vaiioiis aspects aic inteii elated, that is ho say, physiolog- 
ical development mfiiicnccs and is in/luenced by psychological 
and social development Coirespoiidiugly, psychological 
development influences and is influenced by physiological and 
social development, and social development influences and is 
influenced by physiological and psychological development 
Hence, although lesearch in the seveial established disciplines 
helps to identify chaiactciistics of noimal physiological 
giowth, of psychological maturation, and of social develop- 
ment, special leseaich of a co-oidinatcd oi Integiated natuie 
IS necessaiy to establish the desirable balance among these 
seveial aspects of student development 

A second factoi which complicates the foimulation of goals 
foi this field IS the i elation of student peisonnel woik to the 
rest of the college piogiam, In oidei that a college have the 
most effective influence upon its students the vaiioiis phases 
of the college piogiam, ciiiiiciilai and evtiacuiiiciilai, need 
to have some uiidei lying coheience, that is, they must be bound 
togethei by common puiposes. The majoi pui poses of a 
college ale educational, and the acceptable goals of student 
personnel sei vices also should be at least in haimony with the 
educational puiposes of the institution, and piefeiably they 
should seive to piomote these educational puiposes In the 
actual piactice of student peisonnel woik theie is dangei that 
we shall caiiy on activities day after day without carefully 
consideung then i elation to the pnmaiy aims of our institu- 
tion This may lead to a shoit sighted piogiam m which 
immediate goals aie attained without really piomoting the 
ultimate goals of the college It is possible, foi example, to 
woik out a plan of housing which provides foi very quick 
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adjustments of the students to then classmates, and yet by the 
natine of the housing plan, cliques may be encoiuaged and 
the fundamental educational objective of learning to vmdei. 
stand people with vciy difleient backgionnds and to entei 
synipaLhetically into ihe lives of pcisons veiy difleient fioin 
oni selves may be hindeieci lathei than piomoted Oi, a social 
coiinseloi may feel that hci job is well done when she has 
helped to inciease the piopoition of women students who have 
legiilai dates, wheieas the ultimate educational objective is 
to get ri bioadei iindei standing of human bchavioi incUidmg 
a sympathetic understanding of and adjustment to the opposite 
sex Continued dates with the same individuals in many cases 
may letard the attainment of this objective rathei than help 
It It seems necessaiy, theiefoie, to foimulatc goals for 
student peisonnel woik in such a way that they aie closely 
i elated to the majoi educational objectives of the institution 

This implies that the student peisonnel woikei in close 
collaboiatlon with othei membeis of the school oi college 
start will need to examine the vaiioiis types of studies which 
suggest passible goals of student development They will need 
to consider the investigations of the sociologist, the social 
anthi apologist, the social psychologist, the economist, and the 
political scientist) to identify the demands which out cultiue 
makes upon young people and to undei stand the effect of 
cultiual piessiues upon the individual and his gioup These 
studies of the social scientists lepiesent nn important coni' 
ponent fiom which goals foi student peisonnel woik will be 
formulated 

But an examination of icsults of leseaicli in the social 
sciences is not enough It is also necessaiy to examine studies 
of student health and investigations in the fields of physiology, 
nutrition, and psychiatiy— foi these help to claiify the concept 
of desiiable biological development and also to indicate possi- 
ble deficiencies which students may be helped to oveicome 

A third component of research legarding goals foi student 
personnel woik is the held of values Values need to be con- 
sidered carefully not only as possible student goals but also 
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because values, individual and cultuial, condition the student's 
development in many ways The ideals which young people 
absorb fiom contact with the cultviie have a inoie potent 
influence upon student goals, sCiident activities, and the satis- 
factions and disappointments of college life than is commonly 
lealized Any compichcnsivc foimulation of goals foi student 
peisonnel woik needs to considei the values which the school 
01 college may be expected to piomote and the way in which 
school 01 college expeiicnccs may influence these values 

T have suggested sevcial of the stiands which need to be 
consideied in delineating goals foi student peisonnel woik. It 
IS obvious that the selection of goals to be given paiticulai 
emphasis iii a paiticular college depends upon seveial factojs 
One IS the college’s conception of the good life and its 
counteipait — the desiiable peisoii This conception will lep- 
resent not only specific items such as physical health, social 
concern, pcisonal mtegilty, and the like, but it will also involve 
some idea of the i elation of these vaiious aspects At this 
point It IS vciy necessaiy foi the student peisonnel woikci to 
have a woikable but conipiehcnsivc tlicoiy of pcisonality 
stiiictuic and function, and of peisonality development 
Because we do know that the human oiganism shows a con- 
sideiable degiee of unity in its leactions, because we do know 
that physical, social, and psychological aspects arc mtcu elated, 
we realize that one cannot ticat each aspect of a student's 
development in isolation fiom the otheis Some theoiy as to 
how these aspects aie i elated, how they function together, 
and how they may be developed togethei is essential to pro- 
vide a lational basis foi peisonnel work If the student pei- 
sonnel woikei togethei with otliei membcis of the school oi 
college staff has identified moic specifically the aspects of 
human development which the school oi college seeks to pio- 
mote, and if he has a compiehcnsivc theoiy of peisonality 
development, it is possible to foimulate clcai yet comprehen- 
sive goals for his own woik and to avoid ti eating a student 
as though he were a mechanical collection of specific leactions 

What coiitiibutions have tests made oi can tests make to 
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this iUed of research? A test piovides a conti oiled situation 
in which certain specified types of behavloi may be studied 
and ceitaiii phases of this behavioi may be measiued, In the 
effort to formulate a coheient theoiy of personal and social 
developineiitj vaiLous types of tests must be coiistuicted so 
that students can leact in ways which involve the i elation of 
biological, social, and psychological phases of behavioi 
These tests may enable us to see moie cleaily how these 
phases of behavioi are lelated Fuitheinioie, any college, 
after detei mining the tentative goals of its student peisonnel 
work, can employ tests to detei mine which of these goals aie 
of primary importance to its students Conscious attention 
need not be given in the college piogram to those points at 
which students aie alieady developing satisfactouly That is 
to say, tests conti ibute to this aiea of leseaich both in develop- 
ing a compichensive set of goals and in identifying the goals 
which need majoi attention m a paiticulai college at a 
paiticuUr tunc 

A second aiea of icsc*nch in the field of student peisonnel 
woik is the testing of the fundamental bases upon which a 
student peisonnel piogiam is built, A well-iounded plan of 
peisonnel scivices is a recent addition to the college campus 
Most of the schemes have been based upon assumptions which 
have not been adequately tested The piinciples of oiganiza- 
tion, of administiation, of the selection of the staff, of the 
training of the faculty — all aie in need of caieful veiification 
These principles seem to the administiation oi faculty of the 
given institution to be sound, but in many cases they have been 
drawn from fields and expeiieiices which aic not strictly 
parallel to the field of student peisonnel, and it is likely that 
some of these piinciples aie not appiopiiate as pait of the 
foundation of the progiam of student peisonnel seivices 
Research provides a check on the validity of the basic founda- 
tion of the peisonnel piogram. Such leseaich involves com- 
piehensive evaluation of an entue personnel piogiam oi of 
particular procedures 
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A comprehensive evaluation piovides evidence showing 
how far each of the impoitant objectives or goals of student 
peisonnel woik is being attained Since these goals involve 
various aspects of student development, tests of vaiious soits 
aie essential in ordei to find out the points at which students 
aie developing adequately oi the points at which development 
IS Linsatisfactoiy Foi example, this icseaich leqiiiics tests 
of physical development, of health, of peisonal-social adjust- 
ment, of attitudes, of intciests, of skills, of infoim.ition 
acquiied, and the like It also involves a pei iodic piogiam 
of testing so as to estimate the progiess being made by tbe 
students, and correspondingly, then late and degiee of 
development Fuitheimore, an adequate leseaich piogram 
provides a follow-up of students aftei they have been giad- 
Dated fiom college and have gone out into life These follow- 
ups should piobably be made fiom five to ten yeais aftei 
giaduation and should include the collection of data icgaiding 
those objectives which have most peimancnt significance This 
probably would inchiclc evidence legaidmg nitcllcctiml intci- 
csts, health piactices and attitudes, inaiital adjustment, social- 
civic inteiests and activities, and matin ity of aesthetic intci- 
ests Such a follow-up study piovides an unpoitant type of 
data legaiding the contiiuimg development of students and, 
theiefoie, it is a significant phase of the evaluation of the pei- 
sonnel program 

The checking of the fundamental bases upon which the 
student personnel piogiam is built is an area of leseaich 
which has largely been dependent upon valid tests The recent 
acceleiated development of a widci lange of tests has been 
accompanied bv a corresponding increase in evaluative studies 
Tests are making an important contribution to this aiea of 
icseaich- 

A third aiea of lescarch in the field of student personnel 
woik which involves tests is the constiuctlon and validation 
of instruments to facilitate the peisonnel piogiam Tests 
repiesent the major gioiip of these iiistuimenta, Vaiious tests 
have been constructed for use m selecting students likely to 
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benefit from a given college piogiam Much leseaich is still 
needed in identifying impoitant chaiactei istics of young people 
winch can be used as a basis loi college selection and for plan 
ning piogiams of educational and vocational guidance Thus 
far, these tests have laigely consisted of ineasuies of veibal 
facility, of numeiical nianipiilatioii, and of the acquisition of 
mfoimation Tests of higher inLellcctiinl skills, of interests, of 
peisonahsactal adjustment, and of attitudes aie just beginning 
to contiibiite inaikeclly to the selection and guidance woik 
Tests alieady developed have gicatly facilitated the iden- 
tification of students needing special attention, but many new 
mstiumcnts arc also needed Tests aie widely available foi 
identifying certain types of reading difficulty and ceitain types 
of siibject-mattei deficiency New instiumeiits aic needed, how- 
ever, to measine othei types of psychological and social reac 
tion which have fundamental significance foi success in college 
and in life, and which should be identified eaily enough so that 
a piogiam to facilitate development may be begun 

A simiki condition exists with icgaid to tests useful in 
the vocational placement of students Tests of some of the 
essential vocational skills have been of gieat value Tests foi 
identifying ccitain vocational intcicsts aic showing piomisc 
Howevei, some of the fundamental vocational attitudes, 
habits, and ways of thinking have not been cleaily identified, 
noL have satisfactoiy tests lot them yet been developed The 
futiTie contributions of tests of tins type aie likely to be laigc 

New tests aie being constiuctcd to help in evaluating pei- 
sonnel piogiains and piocedutes Judgments of students and 
faculty have not only been supplemented by more caieful case 
studies and obsei vatioiial lecoids, but tests of attitudes, of 
Intel ests, of habits and piactices, of mfoimation and skills aie 
becoming available foi a inoie compichcnsivc evaluation 
Additional tests are still needed, and many aic m the process 
of construction 

I have attempted to suggest btiefly the place of tests in 
thiee aieas of teseaich, namely, m delineating goals foi 
student personnel work, in checking the fundamental bases 
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Upon which peisoiinel piogiains and pioceduies aie developed, 
and m constructing essential instuinicnts foi pcisonnel work 
Tests have ajieady made an impoitant contribution to these 
three aieas of lesearch, but the futiiie coiurilnitions should be 
fai greatei than those of the past The limitations of the 
contributions of the past seem to me to have been due to se\- 
eial ffictois which now can he laigcly overcome 

In the fust place, student personnel woik oiiginatcd 
largely from specific maladjustments within the tiaditional 
college piogiam Pa/ticulai pioblems iclatmg to the conduct 
andmoials of students, then social life, oi then housing led 
to the piovision of special staff membeis to non out these 
difficulties Only within leccnt yeais has theie been wide iccog- 
nition of thebtoad implications of student peisonnel work and 
of the need foi some coheient philosophy and piogiam 
Natiiially, tests used in the student peisonnel field fiequently 
were taken ovci, as they wcie developed, foi othei pm poses 
and used without consideration of the behavioi patterns which 
these tests implied It seems to me that wc aic now icudy to 
foimiilate a coheient conception of peisonal and social adjust- 
ment and to examine possible tests in the light of oui concept, 
discaiding oi modifying tests which do not appiopi lately lit 
this concept and developing new tests that aic in haimony 
with It 

With this bit-by-bit accumulation of peisonnel icsponsibili- 
ties in the college piogiam, it was natnial that the tests used 
should largely be built upon a type of atomistic concept of 
human behavioi, and that the test results should be sum- 
maiized as single scores oi as scpaiate pai ts added togethei 
to foim a total scoie In recent years we have seen moie 
cleaiiy how to construct tests involving gi eater organization 
of behavioi and how to summarize lesults in terms of descrip- 
tive scoics 1 elating vaiious paits of a test, thus getting ,i 
more coheient pictiue of the student’s response This climi- 
nation of the single composite scoic is an impoitant step in 
increasing the contnbutjon tests make to the field of student 
personnel work 
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An fidclitioiifil leason foi niy belief that tests wiU make an 
increasing contiibution to the field of student peisonnel work 
is the wide lecognition of a bioad definition of tests No 
longer aie tests conceived only as papei-and-pencil examina 
tions. Tests aie inueasingly consideied as contiolled methods 
foi obtaining a sample of a student's leactions undei ceitani 
specified conditions With this lecognitlon that a test is a 
means of sampling ceitain aspects of luiiiian hehavioi, atten- 
tion IS now being focused upon cleaiei definitions of those 
aspects ot human behavior which need to be sampled by means 
of tests In educational testing twenty yeais ago, piimaiy 
attention was given to sampling the content of textbooks which 
students weie expected to remembei and to sampling cei tain of 
the subject-mattei skills, such as wilting or numeiical coim 
piitatioii It Js now iccognized that othei aspects of behavioi 
aie impoitaut, such as the way m which the student attacks 
pioblems, the types of inteicsts he is developing, the attitudes 
he has, his lespoiise to aesthetic expeiicnccs such as liteiatuic, 
music, and the aits 

With gieatei claiificatioii of the natuie of testing has come 
a better specification of the behavloi to be tested Twenty 
yeais ago a test in chemistry would be built by specifying the 
topics, that IS, the content to be sampled No conscious effort 
was made to specify the type of leaction the student might 
be expected to make to this content Now we lecognixe that 
we must specify not only the content but also the kind of leac- 
tion expected of the student, the soit of situation in which such 
veaction can be expected and, if possible, the kind of puipose 
which a student would have when i eacting By specifying these 
four aspects of behavior we have a much clearer idea of wh.U 
we are tiylng to test, and this mcieases the piobabjlity that 
we shall control the testing situation sufficiently to piovide 
a satisfactoiy test 
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P ROGRESS in the applications of psychologyj especially in 
the field of aptitude measiuement, will be made by word- 
ing intensively on the measuiing iiisLiiiments which wc alieadv 
have, lathcL than by adding to the laige luimbei of devices 
about which we have insiifTicient research to justify scientific 
application In line with this belief wc have investigated cei- 
tain pioblcms connected with the Minnesota Vocational Test 
foi Cleiical Woikers The poition of this leseaich to be 
repoited heie deals with a noi mauve study of this test 
The usefulness of the Minnesota Vocational Test for 
Cleiical Woilccis has been seiioiisly cm tailed because of the 
fact that norms have been established only foi adults in the 
general population and foi employed adult clerical woikers 
(6) This limitation is the leveise of the more usual and 
seiious one whcie nouns exist foi school populations while 
no adequate noinis exist foi adults and for ciiteilon gioups 
The problem of appropriate norms foi tests Is one of the most 
lugent ones which counselois face in applying measuring 
instillments m guidance progiams wlieie individual analysis 
and diagnosis is an indispensable fust step The Minnesota 


^The coopcmijon of mmiy pornons hni been ncccsaary for (he completion 
or ting study nntl the nulhoi wishes hcicby to express her apprecialloiL Professor 
Donnld G Paier-jon directed (he constajcijon of (his inenfluim? jnstriimeni by 
Dr Dorothy M Andrew and has followed ihroiigli witli ntl^dce and helpful 
siiggesticinfi m subseqvieiit research AasisUtice in Ihc pieparntion of some of the 
mateiinls for this study vvas fiiinished by the pcisonnel of Woik Projects 
Administratian, OITicnl Pioject Numbei 665-7M 69, Subproject Number 229 
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Vocational Test for Cleiical Workeis was standAidi/.ccl on 
adults, and excellent iioims weic developed and lepoited in 
the Bulletins of the Employment Stabilization Reseaicli Insti- 
tute (5, 9) and in the test manual (6) The test with its 
noinis foi adults was used to advantage by the Adjustment 
Seivicc ill New Yoik City, a community guidxince agency foi 
adults, and by othei agencies concerned with the counseling 
of adults As the test has become moie widely adopted, how- 
ever, It has been applied in many situations, especially to youth 
populations foi which the significance of test scoies was not 
known Some woikcis have devised local noims which have 
leflected selective factors of sampling Limitiitions m inteipie- 
tation have necessaiily accompanied limitations in the selection 
of the sample What has been needed is a iioimative study 
of this test based upon a laige sample of youth lepiesentativc 
of the populations at the junioi and senior high school levels 
With such leseaith it becomes possible to apply the Minnesota 
Vocational Test foi Cleiical Woikeis to the age lange foi 
which the test is most appiopiiate fiom the standpoint of 
educational and vocational guidance, 

, The Minnesota Vocational Test foi Cleiica] Woikeis is 
composed of two subtests Test I consists of 200 paiied num- 
bers varying in length fiom thiee to 12 digits. Test 11 con- 
sists of 200 paiied names vaiying in length fiom seven to 
Id Ictteis Slight changes had been made in half of the paiied 
Items and the subject is asked to compaic the paired items 
as lapidly as possible, checking those pans which aie identical 
He IS allowed eight minutes foi Test I and seven minutes foi 
Test II Scoies aie calculated on each of the two subtests 
using the “light minus wiong" foimiila. The administiation 
of the test IS desciibed in the manual and othci soiuces 
4,9) 

The readei inteiested in icscarch evidence of the test’s 
reliability and validity, and in information legaiding adult 
norms and the lelationship between test scores and other vaii- 
ables IS refeirecl to the leferences on page 156 and especially 
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to the monogiaph by Andiew and Paterson (5) The follow- 
ing paiagraphs summarize the lesearch very biielly 

Andrew (5) has piesentcd evidence on icliability which 
indicates that the test yields sufliciently stable icsiilts foi use 
with individuals 

Andiew (5) has also piesentcd a considciable body of 
evidence which points to the Lest as a valuable technique in a 
chnicaJ pjogiani of educational and vocational guidance oi 
selection to eliminate peisons not likely to succeed in clciica! 
tiainmgoi employment The testiesults coiielatc highly with 
high school and college teacheis’ latings of ciciical aptitude — 
in fact, highei than does a test of geiieial intelligence They 
also are definitely i elated to achievement recoids in typing 
and to ciiteiia of pioduction on ciciical jobs as well as to 
siipeivisois' latings of pioficiency on the job The test appeals 
to be mensmmg factois othci than academic intelligence oi 
cleiical tiaining and evperience and it is bettci than otJici 
tests foi clilfeicntiating ciciical woikeis fiom persons in the 
general population 

The lelationship between the two subtests is not siifU- 
cieiitly high to justify using one test alone oi combining the 
two scoies (5) Reading speed is not an impoitant factoi 
in the test 

The method of scoiing is that of 'Sight minus wiong” 
Despite ceitain ciiticisms of this technique (7), it can be 
upheld on logical bases (10) 

Significant sex difieiences on test scores have been lepoited 
(5) for men and women in the geneial population but not 
for men and women employed in the same type of clerical 
positions 

The test authoi (1, 2, 3) has made an analysis of the test 
to dcteiminc the abilities which it is measuung and has con- 
cluded that Test I involves a mimeiical factor and Test II a 
verbal factor and that both aie i datively uni elated to 
academic intelligence, ability to pciceive spatial relationships, 
and dexterity with fingcis and small tools 
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To sccviie uoims which would he faiily icprcseutative of a 
cioss-scction of jiinioi and senior high school pupils m the 
Noith Ceiitijil Association of Secoiid.uy Schools, St Paul, a 
inidwcsteni city of onc-qiiaitci of .i million population, was 
chosen Appioximately 4,000 pupils in giades eight thiough 
twelve wcie given the Minnesota Vocational Test foi Clencal 
Workers Tins does not lepicscnt the entiic population foi 
these giades. In oidei to guard against a possible selective 
sampling in the choice of schools, an attempt was made to 
select at each giade level schools lepiescntiiig the iippei, 
middle, and lower socio-economic groups One high school and 
one junior high school, judged to lepicsent each of these three 
gioups, were chosen To guaid fuUhei against seeming a 
selective samphiig within the schools, the pupils weie tested 
111 English classes, since English is a subject leqiiiied of all 
irrespective of cuuicuium followed. Table 1 shows the num- 
bei of pupils included in the noims, distiibutcd by giade, sex, 
and school Schools A and B lepicseiit the above-avciage 
socio-economic groups, schools C and D lepicscnt the aveiagc, 
and schools E and F weic chaLactciwcd by a huge piapoitioii 
of families in the lowei sjocio-economic gioups 

The testing procediiic was staiulaidizccl and adhciec] to 
thioughoiit the progiam with testing clone in the legulai Eng- 
lish classes The administration of the test was that pic- 
aciibed by the test authoi (4, 5, 6) Peiisonal data items 
including identifying data, date of buth, giade, school, auilc- 
ulum, and father’s occupation weie filled out by the pupils on 
the last page of the test foldei (10) befoie the test itself 
w'as administered Buth dates weie checked against school 
records Additional data, such as high school scholarship pei- 
centilc lank and iiUelligencc test scoies,® weic collected foi 
certain pupils and jecorded on the petsonal data sheet All 
tests wete rescored at least once 

An important pieieqiusite to the publication of noims on 
tests which are to be used widely is a careful desciiption of the 
population on which the norms weie based Only m this way 

^The Aptitude Index of the Vfln Wugchen Unit Scales of Aptitude, Foiins 
E, D, or C, 
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TABLE 1 

lUXIOJ" OP C\SES BY GRADE, SEI, AND SCHOOL 
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Letters designate the different schools 
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aic test coiisuiueis able to detcimuic whethci oi not tt\e norms 
jiic appiopn.ite and applicable to tbeu local populations 
Befoie piesentiiig tho tables of noims ave shall, theiefoie, 
desciibe the sample of the school population to which we 
admuiisteied the Minnesota Vocational Test foi Clerical 
Woilveis, picscntmg statistics on intellectual chaiacteustlcs, 
age-gtade locations, and socio-economic levels 

Table 2 desciibcs a large piopovtioiV of ovii sample at 
each giade level m terms of the ccnLial tendency and vari- 
ability of intelligence lest bcojes Tins table also gives simllai 
ligures foi the total St Paul school population foi these 

I'AHIV. 2 

COMPARISON D1 Till! MJA\5 0> fill’ !>r I'AUI SUIOOl POPUl NHON'' AND A 1 ARCT 



PQRCl m Arii or om 3AM1H V OH TIIl^ OASrS til 

THF UNIT SCAD 5 OF 
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APrirUDE INDEX 
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Did 
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D 

Criadc 

C/i uups 

N 

Menu 

S D 

Dilf 

Dift 


bt Pftul School 
PapuliiDoii 

2,327 

105 3567 

1 1 7982 



VIII 





2 4190 

3 1366 


Cm baiiiplc 

511 

107 7757 

11 7785 




Si P<Kil Sclmol 
Popiilntiori 

2,|C9 

101 7833 

13 73 10 



IX 

Onr SnuH>le 


10« V929 

11 2920 

3 709C 

5 9 DO 


St Pniil School 
Population 

2,299 

10+967+ 

13 1065 



X 

Oiir Sniripk 

716 

106 0964 

12 6935 

1 1290 

2,0610 


St Paul School 
Population 

1,850 

106 8270 

12 2660 



XI 

Our Sample 

758 

106 7081 

12 0660 

B8i + 

1 6866 


St, Paul SJiuo] 
Populnlton 

1,323 

tOB 3117 

11 2010 



Xil 

Oni Sample 

1,0 IS 

107 3470 

U 9260 

^ 9977 

1 +996 


♦These figures foi the St Paul school pppiilntion were piovidcd through the 
courtesy of PvofasgQi M J VuD WflEeDen of ikc Umvcisity of Mmnoaota 


“Ninety three pLi cent of the totnl 3,90+ cftses me so tlesciibctl No inielli- 
gecice teat acoie foi the icroiindcr could he locntcd but there is Do tcason to 
sgspect the opcntion of t selective factor here 
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giades Theie is no necessity that oiii sample should he sLiictly 
lepiesentative of the St Paul population That would have 
been requiied if we had desiied to develop noims appropiiate 
only to the St Paul school population at a paiticulai date 
Oui pill pose has been to develop noims on a sample judged 
to be faiily typical of the school population in Noith Centi'.il 
secondaiy schools and then to desciibc that sample as 
adequately as possible 

It can be seen that oui sample diftcis fioni the St Paul 
school population by fioni less than one to less than foui 
points on the aveiage, depending upon the giade These 
small differences aie more signiffcant statistically fot giacles 
eight and nine than foi giades ten, eleven, and twelve, as can 
be seen from the ratios of diffeiences to the standaid devia- 
tions of those differences Using (liis test of lepreseiitative- 
ness, then, we can say that out tenth, eleventh, and twelfth 
grade students aie, on the aveiagc, moic like the St Paul 
school population fiom which they wcic duuvn than oiii eighth 
and ninth giade students The diffeiences aie small, howevci, 
and It IS not necessaiy foi oui puipose that the sample be 
exactly equivalent to this paiticulai population Fiuthermoie, 
the slight diffeiences in aveiagc scoics on an intelligence test 
would probably not signihcantly altcct the distiibution of scoics 
on the cleiical test which is not measuiing intelligence to any 
great extent 

As a fuither desciiption of oiii sample, Table 3 shows 
the percentage of each age lepicsentcd in each of the five 
giade gioups ^ The age is that at the neaiest hiithday 

A still fuithei dcsciiption of oui sample was obtained by 
detei mining from the pupil’s statements the occupation of the 
fathei and then distributing these oLciipations accoiding to the 
catcgoiies of the Occupational Rating Scale® of the University 

^Ttie render may be mferealed In noting ihc icacmblnnces between thn dis- 
tribution and that which Teiman anti Men ill used for the slaiidaulwation of 
the rctjsed StEnford-Dinet test L M TeimiiJi and M MornlJ, 

IntflUffeuce (New York* IIoughton-Miillin, 1937), ii 17 

'‘Florence L Goodenough and John E Anderson, Expcfunnital ChtUi 
c/(of6ffy (New Yoik Appleton-Centiiry, 1931), pp 501 12 
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TAJlLt, 3 

AGB grade DISTRIDUTIOM IN PPRCPNTAOE5 FOR CASES IMCIUDED 

IN THIS STUDY 


Ciinde 

N 

12 





Age 




13 

1+ 

IS 

Ifi 

17 

IS 

19 

20 21 

VIll 

572 

C 

3S 

+3 

10 

4 

1 




IX 

659 


3 

}9 

42 

12 

4 




X 

7flB 



5 

35 

41 

16 

2 

1 


XI 

SOS 




7 

36 

40 

12 

4 

1 

XII 

1,077 





5 

3S 

38 

14 

3 1 


of Minnesota Institute of Child Welfaic A total of 3,347 of 
0111 3,904 cases weie so dassifiecl fiom occupations as given 
by the pupils whose fatheis weie living, employed in an uihan 
coininunity, and not on lelief Theie wcie 557 cases not classi- 
fied, and these included some foi which the infoimation was 
inadequate The elimination of these gioups from the classifi- 
cation, therefore, tends to give a slightly distoited picture of 
om sample, weighting it loi the vippei socio-economic levels 
Such elimination was necessaiy foi comparative purposes with 
figures available for the United States population and a similar 
mban community, Minneapolis 

Table 4 presents the results of this classification for each 
grade and foi those of oui total sample who were classified 
Comparisons may be made fiist with the distribution foi Min- 
neapolis Our sample appears to be slightly skewed towaids 
the higher occupational levels Pait of this is accounted for in 
the number who weie unclassified Despite that, 4(> per cent ot 
o\u sample have fathers in the upper three occupational 
gionps, and 49 per cent of the Minneapolis male population 
are m thebe thiee groups There aie moie sti iking disci ep- 
ancies, howevei, when our sample is compared with that foi 
the male population of the United States as a whole It is 


150 



P4RENT4L OCCOTATIONS OF CASES IV THIS STUHY DlSTRlBUTTON OF THE KSOMra OCCUPATIONS OF FATHERS WHO WERE LIV13;<?, EMPLOYED 


NORMS lOR MINNESOIA CLERICAL IRST 





151 



inutAllONAJ AND PSVniOlUGICAr MLASURLMFNl 

unlikely that noims developed in a single community Avoiild 
be typical of all communities Noims developed in a single 
locality, when well desciibed, aie moie useful than those 
derived fiom many diveisc populations, a combination of 
which may not be typical of any one situation 

Table 5 piesents the condensed giade norms" foi the 
decile points fot boys and gills scpaiately in giades eight 
thioiigh twelve on the numbei checking (Test I) and name 
checking (Test II) tests of the Minnesota Vocational Test foi 
Clencal Woikcis These noims weve dciived fiom ogive 
cLUves constiucted fioni the distubutions of cases including 
all ages within each gi.ide The numbei of cases and desciip- 
tion of subjects at each giade level have been lepoited earJiei 
in this a i tide The test usei who wishes to apply this test 
to subjects at these giade levels should considei whcthei his 
population is similai to the one used foi calculation of these 
giade nouns 

Some pcisons will picfei age norms, and foi this leason 
we are piesenting in Table 6 age uoims foi these same sub- 
jects who wcie eni oiled rn giades eight tiuough twelve We 
lecommend the use of the giade noims whcnevei possible, 
howevci, as they lepiesciiL actual giadc populations which 
have been dcsciibcd The age noims do not include an entiiely 
lepiesentative sampling at these ages since we included only 
those pupds eni oiled lu school in giades eight thiough twelve 
Fuitheimoie, unequal luimbeis weic selected at the vaiious 
giade levels Actually, howevei, the similaiities between age 
and giade noims are inoie stiiknig than the dilTeiences The 
giade eight noims aie similai to the age noims foi fouiteeii- 
yeai-old pupils, foi example Also notice the sti iking lesem- 
bUnce between the norms foi eleventh giade and seventeen- 
year-old pupils 

In conclusion, it is suggested that the giade norms should 

“Complote percentile norms ate avmlabic m reference tO anrl fiom the teit 
diatnbiitois, The Psychological Corpoi.ition, 522 Fifth Avenue, New York City 
For all pmctical puiposes, however, ihe less tefiticd interpictitions of the test 
scores will be all that are reqiiiied 
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lADLE S 

COHDrNSCD GRADI' NORMS 1 OR D0\3 AND (.IRIS IN CRAllES I I( II f THROUGH IWEIVL ON 
lESrS f AND II OF T»F MINNESOTA VOLATJON II Tl'il lOR CITJircAl M'OSXfiHS 


S(^oie Score 


Deciles 

I CHt 1 

lest II 

Icstl '■Usill 




Giadc VIII 



Males (N 

= 28^) 

FenialcH (N 

= 288) 

10 

1 in 

115 

I6S 

160 

9 

m 

99 

121 

1-Ml 

3 

ifti 


IH 

IDS 

7 

93 

H6 

no 

1 02 

6 

89 

81 

10:i 

96 

5 


77 

Ifli) 

9J 

*1 

8(1 

72 

91 

87 

3 

7fi 

67 

90 

82 

2 

7J 

61 

81 

7H 

1 


^7 

76 

70 

0 


15 

23 




Ginclc IX 



MalcH (N ' 

= 332) 

Fcmalps (N 

= 327) 

10 

15S 

I6S 

ISO 

1«0 

9 

120 

IIS 

iJi 

tJH 

a 

110 

103 

122 

119 

7 

101 


116 

III 

6 

I 9B 

8H 

IM 

11)3 

s 

91 

81 

107 

IDI 

-1 

Sfl 

79 

JOl 

<)5 

3 

HI 

71 

91 

91 

3 

78 

69 

91 

81 

1 

7i 

60 

8J 

71 

0 

^ Jt> 

_30 

_ _ 50 

in_ 



Gj ncic 

X 



Males (N 

= 372) 

Females (N 

= 416) 

10 

165 

170 

181 

IflO 

? 

123 

121 

111 

l-IO 

8 

ni 

J09 

1J9 

130 

7 

106 

102 

127 

121 

6 

lOJ 

96 

1 ’0 

111 

T 

98 

91 

ill 

106 

1 

92 

Hfi 

loq 

nil) 

3 

BG 

HI 

ini 

91 

2 

81 

71 

97 

H9 

1 

75 

65 

87 

79 

0 

55 

35 

15 

Jj 



Grade 

XI 



MrIck (N 

= 381) 

Females (N 

= 427) 

10 

IBO 

195 

190 

190 

9 

ni 

132 

1-19 

1 17 

B 

J21 

1J8 

no 

1J7 

7 

111 

1)0 

131 

129 

6 

106 

10,1 

128 

J3'1 

S 

101 

97 

122 

HR 

J 

96 

9J 

117 

IJJ 

3 

91 

86 

111 

106 

2 

H5 

79 

106 

lOO 

T 

76 

71 

97 

90 

0 

•IS 

^40_ 

55 

_ 55 ^ 



Gindc 

XII 



Males (N 

= 539} 

Fcmillcs (N 

= 538) 

ID 

IHS 

180 

195 

195 

9 

13/ 

111 

151 

153 

8 

I2H 

127 

112 

ns 

7 

IIB 

119 

115 

137 

6 

Ml 

11] 

129 

131 

5 

106 

101 

122 

\2i 

4 

IQO 

9a 

118 

117 

T 

9S 

92 

112 

109 

2 

89 

as 

106 

102 

1 

Ri 

78 

97 

92 

0 

SO 

•10 

39 

10 
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TABLE 6 


CONDENSED ACE NORMS TOR DOY3 AND GIRLS TN GRADES EIGHT THROUGH TWELVE ON 
TESTS T AND II or THE MINNESOTA VOCATIONAL TES1 TOR CIFRICAI WORKERS 



Score 



Score 

Deciles 

Test I Test 11 

Test I 

Test II 


Males (N = 

Age 14 

= 246) Feni'ilca 

(N: 

= 297) 

10 

U2 

162 

162 


172 

9 

114 

115 

131 


129 

B 

107 

103 

122 


118 

7 

101 

91 

117 


111 

6 

95 

89 

112 


lOS 

S 

39 

83 

107 


lOO 

4 

85 

78 

103 


95 

3 

80 

73 

97 


90 

2 

75 

6S 

92 


83 

1 

69 

62 

84 


75 

0 

42 

32 

27 


37 


Males (N = 

Age 
= 323) 

IS 

Females (N 

II 

LfR 

10 

162 

167 

182 


192 

9 

124 

121 

138 


139 

B 

110 

111 

130 


129 

7 

105 

101 

121 


120 

6 

99 

94 

115 


112 

5 

9+ 

89 

110 


106 

4 

90 

S3 

105 


100 

3 

86 

78 

100 


95 

2 

80 

70 

94 


Si 

1 

72 

60 

85 


77 

0 

52 

37 

52 


47 


Males (N = 

Age 16 

= 362) Females (N 

= 411) 

10 

1S2 

193 

187 


187 

9 

127 

135 

US 


146 

8 

IH 

LL5 

137 


136 

7 

109 

105 

131 


128 

£ 

IDI 

97 

123 


121 

S 

100 

91 

118 


lU 

4 

9+ 

86 

112 


105 

3 

88 

81 

107 


100 

2 

83 

75 

100 


93 

1 

77 

68 

90 


84 

0 

47 

37 

57 


47 
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lAULE 6 (Conf) 


Score Score 


Deciles 

Test I 

Tc3tn 

Test I 

Teal 11 



Age 17 



Miifcs (N 

= 433) 

Females (N — 454) 

10 

177 

177 

(92 

182 

9 

135 

137 

150 

150 

8 

125 

122 

140 

140 

7 

117 

112 

133 

132 

6 

110 

IDS 

128 

125 

5 

104 

100 

122 

ns 

4 

99 

94 

116 

111 

3 

93 

SB 

no 

104 

2 

36 

31 

105 

97 

1 

78 

n 

96 

83 

0 

47 

37 

57 

47 



Age 18 



Mnlcs (N 

= 263) 

Females (N = 259) 

10 

132 

177 

177 

172 

9 

135 

130 

152 

150 

8 

1Z4 

122 

142 

139 

7 

114 

in 

134 

ni 

6 

107 

105 

128 

125 

S 

102 

99 

122 

118 

4 

93 

90 

118 

111 

3 

94 

85 

113 

105 

2 

88 

79 

107 

97 

1 

79 

70 

97 

87 

0 

52 

42 

32 

62 

be useful ui junioi and senior high 

schools 

and comnieiCfal 

business colleges which «aic conccincd with the distiibution ol 

then pupils to the coinmeicial classes 

upon the basis of 

aptitude 

foi clerical woik 1 a thei than 

Upon the basis i 

such factois as 

lack of aptitude foi college tiaining. 

This test with its noinis 


should contribute toward a mote scientific ajid wiser counseling 
of pupils based upon all of the peitineiit and available infoi- 
mation Noims foi adults employed in cleiical occupations 
also might be employed in judging a pupil’s clciical uptitudci 
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EXAMINING EXAMINERS 


NORMAN J POWliII 
New Yoik Cily Civil Scnicc Comiitissnm 

T he examination ol applicants and the establish- 
ment of lists of persons eligible foi appointment to pio- 
fesslonal positions in any school system is a most impoitant 
task In New Yoik City, the examining woik is peifon?icd 
by a boaid of seven exainineis selected as the icsiilt of com- 
petitive examination given by the Municipal Civil Scivicc 
Commission In view of the considciablc ament inteiesfc in 
the mattei of exammations for teachei and admmistintive 
peisonnel, a somewhat detailed dcsciiption of the piocediiics 
used by the New Yoik City Civil Seivice Commission in the 
most leceiit test given fot examinci may be of suggestive 
value 

It should be noted that civil seivice examinations, by then 
natnie, are subject to peciiiiai and seiioiis difTicuIties Since 
examinations cannot be lepeated it js not oi flinauly piacticable 
to obtain evidence as to the validity of specific test matenal; 
the selection and use of such matenal must lest laigely upon 
judgment and induect evidence The passing maik is often 
aibitraiily set by law — usually at the 70 or 75 pei cent point 
— necessitating nice judgment on the part of exammeis as to 
the difliciilty of the test matenal in i elation to the calibie of 
the, applicants, if the examineis do not judge the situation 
accuiately, it is then necessaiy to icsoit to the tiansfoimation 
of scores 

Since the examinations aie given as a public seivice and 
the system depends upon public approval, the examinations 
must give the appearance of being just and reasonable, even 
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to the pci son who knows nolliiiig about examinations Finally, 
elements of the examination piocedure aie subject to appeal 
and review by the courts, it must, thcrefoie, be defensible 
beloie judges who know nothing about examination techniques 
No model pioceduie has yet been developed to meet all needs 
and situations The following account piesents one careful 
and painstaking appioach to the specific pioblem at hand 
Adopted in 1937, there is a statutoiy icqiuiement in the 
New Yoik Education Law to the effect that applicants foi 
examlnei positions must be college oi univeisity giaduates 
and possess at least five yeais of public school teaching experi- 
ence To this minimum qualification, the Civil Seivlce Com- 
mission added the further icqiincment of thiee yeais of 
administrative experience in the field of education Applicants 
weie also lequiied to be not more than 49 yeais of age at 
the time of filing application In consequence of a state law, 
only lesidents of New York State weic peimitted to compete 
in the examination 

A total of 114 applications was leceived Of these, 88 
were adjudged as meeting the education, expciience, age, and 
residence leqiniements, 

The IFiHten Test 

Consisting of Dean Ned H Deaiboin of New York Uni 
versity. President Paul Klappei of Queens College and Direc- 
toi Paul M Mort of the Advanced School of Education, 
Teachers College, Columbia University, a special committee 
was designated by the New Yoik City Civil Service Commis- 
sion to piepare the written test ' 

The wiitten test, weighted 4, togethei with an oial test, 
weighted 2, and an evaluation of candidates’ tiaining and 
experience, weighted 4, compiised the cntiie examination In 
order to be allowed to take the oral test, candidates had to 
pass the wiitteu and, in addition, had to pass the oial test to 
be eligible to have their training and expeiience evaluated 
Only 61 of the 88 qualified persons appeared foi the written 

^The looter served as aide to each of the commiltcea who woitecl with the 
exaiDineF test 
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test Thiee applicants withdicw aftci part of the examina- 
tion, leaving a total of 58 candidates 

Divided into font equally weighted paits and with a ni.uk 
of 65 per cent in each pait as well as a geneial written avei- 
fige of 75 pel cent icqiiiicd to pass, the wiittcn test was in 
neithei tiaditionai objective noi essay foim The abilities to 
be measuied did not appeal to lend themselves to usual objec' 
tjve Lest tieatment The precise abilities taken foi mcasiiie' 
ment may be exemplified by lefeicnce to both the questions 
used in the test and the diiections given to candidates 

In Pait I, foi which the candidate was allowed thiec 
hours, the applicant was informed 

"In lating this papci consideration will be given to clnrity in denn- 
Ing the pioblem, cogency of facts used, oulcilv picsciUation of thought, 
conciseness of cxpicssion, niul the geneial effective lies', of the analysis and 
discussion ” 

A single thice-hoiu essay was to be wiittcn on one of five 
pioblems of which the following is illustrative 

"It has been suggested that an examining board concciitcd with 
improvement of its techniques should maintain a research division 

"Analyze tins pioposal discussing the functions, the oiganization, 
the personnel, the values, and the limitations of svicli a division " 

Anothei example is 

"It IS maintained that in examinations for promotion there must 
be full recognition of the contributions winch the candidate made in the 
suboidinatc position 

"Discuss this policy of recognition fiom the point of view of an 
examinei in a school system " 

The second part, lequiiing font horns, consisted of 25 
technical questions The diiections weie similai to those given 
for the fust pait An dliistiative question is 

"A reliability coefficient of 55 can be said to be typical for rating 
personality Units by ordinaiy judgment methods Is this coefficient 
high or low? What basis do you have for voui answer? Wllat diffi' 
cultics aie involved in the interpretation?" 

In anothei type of question leqiiiiing a longer response, 
the candidate was directed to assume the establishment of a 
new^ supervisoiy position in the Depaitment of Education, 
was given the duties of the position, the lequliements for 

159 



] iM't AMDN \1 \KI) i’SM HOlOlrlCAI ASURl MINI 


which UaiI heeii bct up, a\\l\ was asked Lo state whethei he 
.ifriLed with the i cquircnicnts established, whether theie should 
be additional i equnements, and to give a ciitical analysis of 
the statements legaidiiig the oiaJ test to he given 

The foim of Paits III and IV was siinilai In each the 
applicant was told 

“Tins p.iit of the e\niimniition is a test of voin ability to analyze 
t?iven iiioblcm^ to docunieut joui positioti, niul to employ sound icason- 
inn 'I’lic c\ammcis ^\lll he conccincd with join ability to present 
cvitkiiLC, not Midi the natiiic of voni attitudes \ou aie icqunecl to 
tlcnionsti.itc the depth ami hictidtli of vom scliolnjslnp in aiisneuug 
tlicbc questions “ 

Foiii houis weie allowed foi the completion of each pait 
Theie weie 40 Hems in each rypical questions in Part 
JIT aic 

“5 'ICdiication is a phase of Livili/atioii, not the whole* What 
me the implications of tins stati'nicni foi cclucation as one 
of the soLict.il Agcniics^ 

“9 ‘No philosophy of education is fundamental until it is based 
on sociology — not on physiology, not even on psycholagy, but 
on sociology ' Is this ,\ valid statement? Why ? 

“15 Is it possible foi ctliicaliun to be non p.iitls.in^ Whv^“ 

lixampLcs of the questions in the foiuth pAit ,iic‘ 

“5 ‘The VTiy fact of contact hclw ecu two ciiltiiics tends to cii- 
gciulci fcatiiics iicM to both ’ IVJiat basis is tlicic foi tins 
statement? 

“17 ‘IncitiA conditions the solid fiamcvioik of society and makes 
cultiiic possible* Is this a valid statement? Why? 

“12 'As a giQup, the aged arc mcvcasinG fastn than the general 
population’ Enumciaie thicc highlv significant social 
effects “ 

Au elf 01 1 was made to eliminate the deficiencies of cus- 
tomaiy essay testing and to intiodncc the majoi advantages 
of the objective test, while letaining the vii tries of essay test- 
ing, It may be pointed out that the coirsideiable length of 
the examination made possible both intensive and extensive 
Sampling The type of item used iti Parts TII and IV has 
been subjected to quantitative analysis by the Social Science 
Research Council with the finding that it is an “extiaoi dinarily 
useful” instrument Dr Brigham is of the opinion that 
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the kind of e^um^natlOll question iitili/cil in llie thiul mul 
foijith paits of the test mc.isiiies ‘'iMc.ultli ol hacki^iouiui;' 
though the ditectioJif m the lest undei considci .itJon here 
stjite thfit depth of schohuship is tiiso to be ckiiuMisti.itcd " 
Ceitfiinly Pait I appioaches closely the .ippi.ns.il oi depth 
aspects and Pait H piobcs depth to a somewhat Icssci dcgice 
than it evaluates bieadth 

The cential difiiculLy involved in essay examinations is 
iiiiieliability of inting To pi omote objectivity in lating the 
Uht two pa its of the lest, the extent of the i espouse hy eandb 
dates was coiibtiictcci tempoially and tlicicfoic spatially A 
seven-point lating scale was employed in which the chaiae- 
teiistics of best, niediocie, and pool answeis weic lecoidcd 
to seive as guides foi the aw.uding of ci edits IXhniie key 
answeis weic foimulated to all the questions in Paits I and 
II of the test and a scale foi the allocation of ci edits was bcl 
up In all pnits of the test, lating keys weic constiucLcd hy 
two 01 tliiee exanuneis iii confeicncc, pai tially hv lefeicncc 
to lelevant litciatme oi othci souiccs, paitially by examin- 
ing candidates' answcis to pi ovule a icalistie seining basis 
Maikingwas peifoimcd independently by two oi tbicc exam- 
ineis who, aftci the completion of the scoj mg, compaied then 
latings. All disci cpancics except those of a ttivial ch.uactci 
weic noted and caiululatcs^ aiiijweis wcic icicad to find an 
equitable base foi agi cement by the laleis as to the maik 
meiJted by the specific answei Final laiings weie the mean 
of the individual cvamineis’ mai ks 

Foi the type of examination employed in PaiLs ITT and 
!V, a lating lehability of 87 foi total scoic has been found 
foi a 50-qiTcstion, /-oui-houi social science test’’ In a 40-qucs- 
tion examination of similar foim given foi piomotion to 
Captain, Dcpaitmcnt of Collection, New Yoik Cityi the coi- 
1 elation foi total test scoie between two ratcjs was ,93. Split- 
half reliability adjusted by the Spcainian-Biown foi inula was 

■ ■ -i 

"C C Hfighjim, I’elhws/np i/iftlicimls (Pnnt.c(on Pnnccfnti 

Univei'^iw Picss, 1535), pp 22-3 

’'/ ffui . ]) 1 1 


161 



IDUCAHONAl \N1) PbYt’llOl.OCJK’M ML* AbUWLIvirwr 


92, while the staiulaid ciioi of incasuiemeut was 3 30/ 
There appears to be faiily substantial evidence that the kind 
of examination constuicted la satisfactoiily leltable, 

Much of the basis upon which the widespiead belief m 
essay test uiuehabdity icsts seems to be a deiivative of expevi- 
mental findings aiising fioiii biased invcbtigatioiis The bias is 
a LCsult of lating without the use of keys so that cliffcLences 
between rateis aic diifeienccs between judgments as tt> the 
nature of correct icsponses and the magnitude of the ci edits 
to be awaided to paitially collect answcis as well as divei 
gcncies in the appraisal of paiticiilai i espouses It appeaia 
exceedingly probable that there would be diffeiences of opinion 
among expeits in the lating even of many multiple-choicc 
questions if the experts weie not piovlded with a scoiing key 
In the present instance the foimulation of key answeis and 
lating scales, fieqiicnt conlciences among lateis, and the use 
of several raters tend gieatly to eliminate unieliabdity of rat- 
ing in each part of the wiitten test 

The essential, significant chaiacteiistic of a test is its 
validity Reliability is only of incidental impoitance since a 
test may be leliable without being valid but cannot be valid 
unless it is also reliable Unfortunately, in the written test as 
in the 01 al and expciience measuies, it is not possible to com 
pute a validity coeflicient m teims of an acceptable cutciian 
of ability on the job No satisfactoiy ciitciion exists, only 
one candidate was appointed subsequent to the examination 

A validity judgment may, howcvei, be piedicated upon two 
elements, the backgrounds of the special examining conimittee 
and the reasonableness of the appeal ance of the examination 
Both facto IS suppoit the belief that the wiitten test la valid 
The examining panel consisted of piominent educatois highly 
experienced in the selection of pcisonnel Also, the wiitten 
test ranged widely over many subjects appaiently pertinent 
to examining woik, and its length was sufficiently great to 
make validity an exceedingly probable attiibute of the test 

‘Bureau of Research, New York City Civil Service Commission, ^'Selcclion 
ot Captains in iht New York City Deparinicnt of Correction,” Piilrhc Personiul 
Qtiarterl}), I 1, 6-7 
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The final factoi of gieat impoitance is the diffeicntiAting 
capacity of the test In tcims of maxima of 100 pei cent, 
the applicants’ scoics aie set foith below 



Port I 

PariH 

Part m 

Pail IV 

Mean 

1521 

S3 7 

+7 1 

50 0 

Siiindafd Deviation 

15 1 

11 6 

12 8 

100 

Highest Score 

HG 

7M 

68 6 

77 1 

Lowest Score 

iS 0 

26 3 

214 

25 7 

Range , 

59 6 

<VB 1 

17 2 

SI 1 


Test scoies sepaiate well among candidates Applicants 
me distributed over approximately 50 pciccntage points, about 
half the total possible lange It is the middle half of the scale 
which IS occupied by candidates’ scores The langc is loiighly 
from 25 to 75 pei cent except foi Pait I whcie scoies avc 
distinctly higher The highest iiiaik foi the wilttcn test com- 
bining all foul parts was 76 7 pci cent The passing maik 
was 75 per cent 

It follows that citlici the wiitten test was too dlflicuU or 
the candidates weic too pooily equipped, Either conclusion 
suggests the desii ability of tiansmuting oiigmnl scores into 
highei marks, If the test was too difficult, some of the faihiics 
should be passing pci sons If the applicants aie defective, the 
condition is imfortimatc but laigcly the pioducl of the ligicl 
statiitoiy lequiiements limiting applicants to paiticulai gioups 
An extensive publicity campaign had insured that all oi prac- 
tically all qualified pei sons were aware of the oppoitumty to 
compete m the examination 

The adjustment of maiks involves the necessity for detei- 
mining the natiue of the tiansmutation piocess. In each test 
part, the mean was taken as the point of reference and denomi- 
nated 75 pel cent Distances in standaid deviation units above 
find below the mean fixed the ptccise pcicentages awarded to 
candidates. 

Thus, of the 58 candidates, 29 wcic passed in the written 
test The pm pose of the examination was to place on an 
eligible list those pcisons who appealed to be qualified foi 
the position of cxaminei With this guiding piinciple in mind, 
It was considered dcsiiable that the better half of the candi- 
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clatei, tht wiitLuii test be (^ivcii the o\}poitmiitY to sub- 

mit to fiuthcj c\annn{\tion It was coiisitleiecl that all those 
111 the iippei half of the gioup had demoiistiated the posses- 
sion ol a coinpmatively .uccpublc nniiimiiin of scholarship 
Rcicaliiig, then, involved taking one point of icfeience msteatl 
ot iinothci It was bebeved that incompetents who managed 
to slip by in this pioccss would be cnuglu in the tests winch 
wcie to follow An oial test then was administcied to the 29 
peisons who had siuvived the wiittcn 

The Otnl Test 

The test was given in two p.iits equally weighted and 
scpaiatcd in time by about si\ weeks The fust pait was 
designed to measme technical competence, the second set out 
to appiaise Judgment, clearness and quickness of coinpieheii- 
sion, mannei, appeal ance, and speech 

In Part 1 of the tcchmcal-oj al test, the 29 peisons who 
had passed the wutten examination weie divided into six 
gioups, five with five peisons and the sixth with foui peisons 
Roi each gioup a demonstiation oial examination was 
enacted The demonstiation oialsweic ostensibly given foi a 
paiticiilai job 7'he paiticulai positions foi which the demoii- 
stiationb weic held wcie tcachei of English in the high 
schools, teachci of economics in the high schools, psychologist, 
leseaich assistant, elemental y school principal, and director 
of adult education 

One gioLip of candidates obseived one demonstiation, a 
second gioup obseived anothei, and so on In each case both 
demonstiation examiner and subject weie members of the 
examining division of the Commission Into each demonstia- 
tioii ceitain defects and virtues were intiodiiced both with 
regaid to the demonstiation e\ammei and the demonstiation 
subject Demonstrations were written and planned in advance 
Candidates weie icquued to i.ite both paiticipants in the Intei- 
view which was enacted Each demonstiation lasted foi about 
one-half houi Candidates xveie pei nutted to take notes while 
the demonstration was in piogiess and then allowed an addi- 
tional fifteen minutes foi note taking 
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Following the ciemoiistisiLion, caiicliclatcs ictiieci to an 
adjacent loom and weie summoned individually foi an oial 
examination befoie the examining panel This oial examina- 
tion lasted foi not less than one-half houi 

Candidates wcie latecl by the panel in accoi dance with a 
set of diicctions which had been foinuilated in advance and 
jn accoid.ince with ciiLeiia picp.ucd piioi to the demonsUa 
tion and adjusted aflcj the demonsti ation to 111 the pci foi ni- 
ance which had been obseivcd 7'lie riieinbcis of the panel 
viewed the demonstiation at the same lime as the candidates 
in 01 del to be able to adjust nccii lately the ciiten.i foi in ting 
to accoid with the demonsti ation obsei vetl by the candidates 
The ratings leceived by candidates wcie cletciiiiiiicd by the 
lutings they had given to paiticipants in the demonsti ation 
niid by the adequacy of the snppoiL they wcie able to adduce 
foi tJiEii la tings The candidate’s evaluation of the clcnioii- 
stration examinci was leqiiiicd fo be siippoited by obsciva- 
tions on the cxaminei’s attitudes towaicl the subjecti his skill 
in questioning, and the general conduct of the inteivicw The 
evaluation of the demonsti ation subject was leijuiicd to be 
siippoited by obsei vations on speech, mamici, judgment, and 
appeal ance 

Of the 29 who had taken Pait I of the oial, 16 qualified 
to pioceed to the second pait That an applicant was “quali- 
fied to pioceed” did not necessauly mean that he passed Pait 
I, since a geneial aveiage of 70 pei cent in the oial test as a 
whole was lequiied in oidei to puss Foi example, candidates 
who obtained 60 pei cent m the hist pait of the oral pio- 
ceeded to the second pait of the oial with the possibility of 
passing the entiie oial test only if they obtained a scoic of 80 
per cent on the second pait, which would give them the 
lequired aveugc of 70 pei cent The maiks leceivcd by the 
16 candidates who qualified in the fiist pait of the oial weie 
Maik Fiequency 

60 0 , ... , . 3 
60 8 ... ... 2 
62.1. ... . I < 1 
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3 
1 
1 
1 
1 

1 

2 

Only tluee peisons obtained scores of 70 pei cent or bettei 
in the fiist part of the oial It is indicatedf then, that the 
large majoilty of the candidates who appeared foi the second 
part of the oral had alieady exhibited mediocrity with regard 
to technical competence^ 

Theie is a sizable disci ep.incy in score between Part 1 and 
Pact TI in only two cases. In one case, a candidate who had 
obtained 75 8 in the first pait icccivcd 60 8 in the second In 
the olhei case, a candidate who had obtained 74 2 in Pait I 
icceived 59 0 in Pait II The point of these data is that can- 
didates weie consistently pool The fact of the matter is that 
most of the candidates pei formed in mechocie fashion in the 
fust pait of the oial and merely confiimed theii mediocrity in 
the second pait of the oral, already having shown mfciioiity 
in the wiitteii test 

The 16 Individuals who took Pait II of the technical-oral 
test weie divided into eight groups of two For each group 
of two peisons, examined separately in a particular half day, 
two types of situations weie set up In the hist type of situ- 
ation, the candidate was dnected to assume that he had been 
appointed to the position of examinei and that he had been 
seiving m tins position foi about five yeais The candidate 
was told that he would be visited by a pel son with whom he 
was to talk, for about half an hour and that he was to conduct 
this interview as naturally and effectively as he could Foi this 
examination, one person, Professoi Robert K Speer of New 
York Univeisity, acted as the visitor and assumed a diffeient 
role foi each group ot two candidates The loles assumed 
were lepresentative of a paient association, assistant exam- 
inei, colleague on the boaid, reporter, representative of the 
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Civil Service Commission, visiioi fiom Sionx City Intciesteci 
in teaching peisonnel, failed candidate, and lepiesentntive 
of a teacher tiainmg institution 

A second type of situation was established after the con- 
elusion of the candidate’s intciview with the visitoi This 
consisted of tliiee oi moic questions M\nmpJes of the fiist 
kind of question aie 

Can education be iceoiistiuctcd thiougli uscnrcli? 

How would tile adoption of a paitiaihu pbilosopliv of c\aminlii[i 
m New York City affect educational practice and tliinking throughout 
die whole country^ 

What do you eonsidei to be the majoi viitues (or defects) of our 
educational system in the United States^ 

Foi the second question, the candidate was given a quo- 
tation, asked to tell whether he agieed oi disagieed with the 
quotation and what the implications wcic of his agreement 
01 disagieemciit for the work of the Boaid of Examineis in 
New Yoik City Some typical quotations aic 

"The dangci which conics fiom cmphasi/.ing the significance of 
contenipoiiuy changes is that Imsty and ujisoiiiid icvisiuns will be made 
in the cuiiicuhim ” 

"III the solution to educational piobkms lies the solution to all 
social pioblcms/’ 

"The mam function of education is to perpetuate deniociacv *’ 

"Adult education should be liinitcd to cducablc adults " 

Foi the thud qiieshon, the candidate was required to talk 
for seveial minutes on any topic which he deemed to have 
implications foi the woik of the Boaid of Examincis This 
was followed, wheie appropiiate, by having membeis of the 
panel question the candidate directly in order to obtain, 
clarification oi amplification of one or more points made by 
the candidate It js noted that members of the panel weie 
fiee to ask questions of the candidate at any time in order to 
explore a statement by the candidate 

The diiect questioning by the panel was introduced by 
having the candidate talk foi several minutes on his experi- 
ence and backgiound mainly in older to give the candidate an 
oppoitiinity to “waim up” piioi to the questioning The rat- 
ings given to candidates weie made in accoi dance with wiit- 
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ten dll ections adopted by the examining panel In the situation 
in which the candidate assYimed that he was alieady an e\am 
inci, the following rating ciitciia wcie employed soundness 
of position taken, cogency of discussion, claiity of discussion, 
penetration of tieatmcnt, lime taken foi effective oiganwation 
of 1 espouses to the visitoi, niannei and attitude adopted to 
waid the visitoi, quality of speech, and appeal ance In 
the situation involving moic diicct questioning, the follow- 
ing ciiteiia wcie used impoitancc of mateual selected, sound- 
ness of position taken, lelevance of mateuni selected, clarity 
of piesentation, penetration of tieatment, quality of speech, 
manner and attitude adopted towaid the panel, time taken for 
effective oig.ini/iition of lesponscs, and appcaiance 

Infoimatioii foi the lating of the five factois was supplied 
both by the cliicct questioning and the assumed examiner 
situations, In the lating of the five factois, a rating scale was 
employed which langed fiom 0 to 100 per cent The stand- 
ards set were 

"Unaccepuble caiidulitcs dioultl be given latings below 60 per cent 
R.ituigi between 60 pci cent 'iiid 75 pci cent should be given to candi 
dates whose cliaiacteiistics consulcied in this pait of the test are only 
very slightly infciioi in level to those of a lugh-ginde cxamiiiei Ratings 
above 75 pei cent should be given to candidates who undoubtedly 
possess a high level of the chnractcnstics defined in this part of the 
cvamination ” 

There weie three examiners in the fiist part of the tech- 
nical-oral. Joseph G Cohen, Dnector of the Division of 
Graduate Studies, Biooklyn College, NedH Dearboin, Dean 
of the Division of General Education, New York University, 
Maigatet V Kiely, Dean, Queens College 

Because the second pait was legs susceptible to objective 
latlng, the examining committee was incieased to five in 
Older to miiiimixe subjectivity Ned IT Demborn of New 
York University; Willaid S Elsbice, whose special field 
IS teachei peisonnel, of Teacheis College, Columbia Univci- 
sity; Margaiet V Kiely of Queens College; Jesse H, Newlon 
ot Teachers College, Columbia Umvcisity, Oidway Tead, 
Piesideut, Boa id of Higher Education in New Yoik City, 
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The tiaditional deficieiKics of oiai tests aie well known 
and include in civil seivice exnnruning the tlilTiculty of nchicv- 
mg both the tact and the appeal ance of satisfactoiy icliability 
and validity Appeaiances are of consideiable impoitancc in 
public peisonncl admmistiation. Not only must the ex.vmin,i- 
tioii be an eftective instiiinient but also iL must avoid the 
impiession of being aibitiaiy, iinfaii, oi capiicious even 
though It IS none of these in fact The difiiailty is gencially 
that of describing adequately the basis for latings and of 
connecting clearly the lating scale with the candidate’s pei- 
formance m the deteimination of maiks. It must be pioved 
that marks aie acciiiate and unbiased. This necessity was lec- 
ognued and met by setting foith in writing the natiiie of the 
scoring scales, ciitena, and standaids used, and by keeping 
stenotype and phonogiaphic lecoids of all questions and 
aiisweis The cffoit to have the examining panels be lathc; 
laigc and lepiesentative of divcisc educational viewpoints 
and to have them consist of leadcis in the piofcssion was 
also consideicd to contiilnite towaid the objective of coupling 
seeming with actual validity 

The technical requisites foi icliabihty seem to be present 
The aveiage mteicoii elation of the exannnets’ ratings on the 
first part was 91, making an estimated reliability for their 
composite latmgs of 97, by the Speaimaii-Brown formula 
The average inter coi relation among the exiimineis on the 
second part was 785, making the estimated leliabiUty foi 
then composite ratings 95, The average difference among 
examiners m Part I of the oial is 2 2, in Part TI the aveiage 
difference is 5 4 Scoies were in five-point units as 50, 55, 60, 
65, 70, so that the disparity in giades is about half of one 
point on the Part I lating scale and about one point on the 
scale in Pait II. 

Since quantitative appraisal of the validity of the oral 
tests IS impossible, the problem must again be approached 
logically Validity lefeis to the degiee to which a test meas- 
ures what it sets out to measiue The oiai attempted to 
evaluate ability to judge applicants for educational positions, 
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to analyze weaknesses and stiengths in oial examining meth- 
ods^ to deal with visitois, to display good judgment and com- 
prehension, and to exhibit a satisfactory appeal ance, mannei, 
and speech Situations weie formulated with the explicit pui- 
pose of measuring these factois^ all of which appeal to be 
significant samples of the examining task, so that fiom the 
viewpoint of job analysis the oial appeals to be acceptably 
valid 

The Co?Nbnied Scoies 

When the scoies for both paits of the oial weie combined, 

It was found that only one candidate had achieved a passing 
rating At this point in the examination, howevei, adjustment 
of marks to pass a greater number was deemed undesirable 
There are several leasons foi not tiansmntmg marks in the 
oial test. In the oral, the identity of candidates is known In 
the written, identity is concealed by having candidates enter 
then application numbcis in place of then names “To tians- 
mute maiks vvheie identity of applicants is known is to make 
possible the charge of manipulation Furthci, the wutten 
was followed by othei tests able to weed out the unfit, the 
oral was to be followed by an experience test in which eveiy 
peison admitted to the examination was ceitaiii to leceive a 
passing mark because all possessed the pi escribed minimum 
education and experience qualifications Moieovei, veiy sub 
stantial opportunity had been afforded to aspiiaiits foi the 
position of examine! to prove themselves Applicants had 
been examined at foiu sepaiatc occasions in the written test 
and at two different times in the oial Finally, the mattci of 
standards is highly relevant in deciding whcthei oi not to 
rescale marks 

The position of exaniiiiei is of the utmost impoitance in 
a school system The examinei is lesponsible for the selec- 
tion of educational personnel and therefore, in a laige meas- 
ure, for the quality of the teaching done and the manner in 

“TKe prncUee of the New York Ctty Commiaalon is to affix latiitg numbevs 
to nil written test papers and to detach, the application numbei from answer 
sheets The applicant knows his application numbei, but not his rating number 
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which the youth of the city is taught and molded The posi- 
tion pays $11,000 a year and is held foL life after a six-month 
piobationary period which is not made effective since no ap- 
pointee to this position has evei been dischaiged after pro- 
bationary appointment It also must be home in mind that 
educational piactice in New York City aftects to a degree 
educational piactice In the remaindei of the coiintiy It seems 
leasonable to believe that under these conditions a high stand- 
ard IS desirable for this position The position was taken 
by the Commission that passing only one of 58 persons taking 
an examination of this type is not evidence of an unjustifiably 
high set of standards when the niimbei of jobs to be filled is 
very small 

A great low arose aftei the eligible list of a single name 
was published It would be inteiesting and instinctive to take 
up the contioversy in detail, hut such a discussion belongs 
elsewlieie Some of the objections can be laid to a lack of 
uiulci standing of fundamental mcasuicment piinciples. 

The examination was leviewecl thiee times by the couits 
and once by a committee on manifest ciiois established by the 
New Yoik City Civil Service Commission First of the many 
and varied inteipretations of the data came with the appoint- 
ment of the committee on manifest errors to hear and judge 
candidates' appeals The usual pioccduie of the Commission 
IS to refer all appeals to a board of three mcmbeia of the 
Commission staff In view of the Importance of this examina- 
tion, howevei, and the necessity of eliminating any suspicion 
of bias, the special panel was constituted Its pcisoiinel con- 
sisted of Arthur A Ballantinc, noted lawyei and Undersecie- 
tary of the United States Treasuiy under President Pier belt 
Hoove i , Charles J Piepei, Professoi of Science Education 
and head of the Department of Science Education at New 
Yoik University; William F» Russell, Dean of Teachers Col- 
lege, Columbia Univeisity In a report dated May 3, 1938, 
the Commission found “no manifest eiioi in the examining 
methodology or in the constitution of the examining panel," 
stated that “the pass maik was set neithei too high noi too 
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low in 1 elation to the level of competency leqinied/’ and con- 
cluded that there was "no manifest erior in the i.Uing of any 
candidate " 

Suit to invalidate the test was then bi ought by seven of the 
failed candidates It was held by the New Yoik Suptcaie 
Court "that the techiiicaUoial test against which the piincipal 
assault was made was meticulously prepaied and impartially 
admimsteicd, that every safeguard to insure fairness and 
equality of competition was piovided, and that the standaids 
used in rating the competitors were in legal contemplation 
objective and leviewable,** 

The failed candidates had grcatci success with the State 
Appellate Division Five justices of the Appellate Division 
concuiicd in finding the oral examination invalid The justices 
disagiced quite strongly in legard to the selection of the 
ground upon which to icst then conclusion One stated that 
it was illegal to limit the eligible list to one name; a second 
was impiesscd with the "compaintive incompetence” of the 
sole passing applicant, others intetpictcd the evidence to point 
to the intiusion of ideological considci ations in the tcchnical- 
oral test 

The final wo id came with the decision of the Court of 
Appeals which disagreed with the Appellate Division as to 
why the oral test was illegal but agieed that the test should 
be given all over again 

Following these vicissitudes, the New York City Civil 
Service Commission held a new oial test in 1940 This time, 
three candidates weic passed, The applicant who had been the 
only one to qualify in the pievious test was included among 
the three who were successful m the new one, 
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r R SARDiN AND L, S 1J{)RI)1N‘ 
Universily of Minneiola 


I F ALL the liteiatuie on the piedlction of college giades 
weie to be assembled In one place, the outstanding chaiac- 

tenstic would be the almost iiniveisal agreement that con ela- 
tion coefficients highei than 70 aie practically impossible with 
existing methods As a mattci of fact, Segel has collected 
ovei a bundled such studies only to dlscovei that the median 
piedictive validities of high school scholaiship, tests of gen- 
eral achievement oi aptitude, and tests of specific aptitudes 
01 achievements weie 54j 44, and 37 lespectively^ 

In studying the factois which aie lesponsiblc foi these 
lelatively low coefficients, oiii attention is immediately focused 
on the nature of the ciiteiion — the honor-point ratio Com- 
monly used by colleges and iiiiiveisities as an index of the stu- 
dent’s achievement, this summaiy figiiie icpiesents attainment 
in many diffeient kinds of courses taught by various kinds of 
teachers with different standaids of measurement. 

Two characteristics of this criterion are of importance for 
predictive efficiency, namely, its unreliability and Its heteroge- 
neity The fiist chaiacteiistic, unreliability, has not really 
been measured effectively, but can be estimated by logical 
analysis It is agreed that even with improved methods of 
measuring attainment in college courses a semi-intuitive, hit- 

^We aie indebted la Piofessor E G Wdlipnison for atlnwlation and advice 
in the formulation of this papei We gratefully acknowledge hjs pcrmiBkion lo 
iiae port of the data contained m Kis study Predichon 0/ Sacass ui ilie Arts 
College to be published in bulletin form by the University of Minnesota 

^Dayid Segel, PredicHoii of Success m College (U, S Office of Education, 
Bulletin 1934, No IS), p, 70 See also Daniel Ilarns, “Fnetora Affecting CoU 
lege Giadts A Review of the Litciature, 1930-37," PsHliohg\cfll Bulklin 
XXXVII (1940), 125-66 
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01 -miss judgmental factoi still remains in the grading process “ 
That this would create a mcasuie of luirehabihty In the indi- 
vidual coiuse grade is imclemable As long as each teacher 
has n set of standards, individually deuved and reflecting 
somewhat iiniQiie set of objectives, so long will giades retain 
their unreliability When we compound the umellabilities of 
the individual coiiise grades — Avhich wc do in computing honoi 
point ratios — it is impiobable that the lehability of the final 
(.ritenon will approach the lellability of the picdictois 

If It weie possible to establish peifect reliability of course 
giades in individual subjects and of the honoi -point ratio, the 
second characteristic of the criterion, heterogeneity, would still 
lemam to inteifeic with piediction Foi students who aie 
taking courses in natiual sciences, mathematics, social sciences, 
and languages in Viuying combinations, the ciiteiion repre- 
sents a complex of many factors each of which logically ought 
to be sampled by the components of the piedictlve batteiy It 
IS self-evident that the moic complex and lieteiogeneous the 
factois in the criteiion, the moie difficult becomes the task of 
assembling a pieclictivc test battciy which will adequately sam 
pie this aggicg.ite without simiikaneoubly intioduung into the 
predictive index other extiancous factors Oui task would 
be solved if we could assemble a senes of puie measuies foi 
each component in the criterion. Pine tests, howevei, have 
not yet been created The early promise of the factoi analysts 
that a pine test was possible has not yet been lealizcd 

A word of caution is in order foi those who would hasten, 
aftei having discussed the unreliability and heterogeneity of 
the prevailing criteiion, to do something about it The ume 
liability or reliability of a criterion is only one factoi in pie- 
diction. Of equal importance are the reliability and validity 
of the piedictive battery As already indicated, techniques 

"This js not (0 imply that jiidgmenla are to be abandoned J3y learning to 
avoid the pitfalls and fallacies in human juclgmenta, teachers can improve the 
quality and consistency of their ratings Several writers hove treated at some 
length the common errors in miking judgments See H E Burlt, Prtncipifs 
af F/nploytneftt Fjyeho]offy ^Boston Hough ton-MiiRin, Chapter 11, and 

M S VueleSj JTtdujtnal PsycAoloffv (New York* W W Norton Company, 
1912), Chapters IX, X 
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have not yet been developed for ci eating tests which wjll he 
pure measiiies of any single factoi, Thui stone’s utilization 
of factor methods in his Piiniaiy Abilities tests has not yet 
passed beyond the experimental stage In fact, hist lepoits 
have been conflicting * While the leliabilities of the picdictive 
tests such as the Ameiican Council and the Ohio Psychological 
Examination distiibute aiound 90, these alone do not oifci 
hope foi a gieat deal of improvement m validity 1'luis, 
reseaich Ingenuity must be applied to the piedictoi Viuiables 
as well as to the criterion 

One final limiting aspect of prediction must be taken into 
account by the research woiker befoie pitching his aspiiations 
too high This is the indeteiminancy principle that I'leiscn- 
berg has formulated for piediction in the physical sciences 
Present-day thinkers iccognizc that spontaneous and uncon- 
trolled factoi s aie always present These cannot be foieseen , 
they will introduce a measuie of erioi in any foiecast, Among 
such factors to be found in the prediction of academic achieve- 
ment ale momentaiy motivations such as health coiuliLioiis, 
social distj actions, sexual distiactioiib, home conllicis, tem- 
porary moods, sets, fatigue, and so on Because of these not 
readily controllable elements, it would be safe to guess that 
even with peifcctly reliable criteila and with statistically 
infallible picdictive tests, the uppei limit of multiple coiicla' 
tion would still not exceed 95 But such a pessimistic outlook 
need not be discouraging to fiuthei research Much loom 
remains for improvement The increase in picdictive efficiency 
of a correlation of 70 to one of 95 lepresents a range of 
about 4 1 per cent improvement over non-test estimates 

The ciiix of the pioblem of selection and admission of 
students hinges upon accuiate prediction instuiments Pre- 
diction selves the piupose of assisting college authoiities to 

*J M Staltiaker, ‘"Purmiy Meuta! Abilities, ’* Sc/iOQl am/ Society, L (19l0), 
868 IZ See also R G Ileritrciiter, '‘Primary Ability Tests Applied to Engi- 
neering Preahmen," Ptyc/iohfficd Bulletni, XXXVI (1939), S4B-+9, nnd WillUm 
Ml Shanner and G Frederic Kuder, “A Comparative Study of Frcsbmnn Week 
Ttsta Given to the tfniversily of Chicago/* tducfltiona/ and Psyc/iotofficai 
^fe^Ilure^netftJ I (1941), 85-92. 
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select stiulciits who have a leasonahle chance of piofiting fjom 
the college’s oflciings Since the etforts of the test-maktrs^ 
educational psychologists, and othei icseaich woikers i cached 
a ceiling at foiecasting eflicjeiicy of approximately 28 per cent 
hcttei than non-test picdlction, furthei leseaich may take any 
of thiee couises 

1 furthci improvement in the jeluibility and validity of 
the pjcdjctivc battery, 

2 impiovement in the ichablhty of the cnteilon 
measuics, 

3 design of a new ciiteiion which will be moie predict- 
able and at the same time acceptable to school 
administiatois 

At the piesent time, the first appioach appears to be the 
one least likely to htmg success, yet it is the one most fre 
quently selected, With the development of the method of factor 
analysis hopes weie laised foi a significant increase in the 
efiiciency of tests The belief pi evaded that with the isolation 
of factois in a test batteiy the foundation might be laid foi 
the constuiction of puiei tests which in tnin would lead to 
moie acciiiate piecViction Thus fai this piomise has lemaincd 
unfulfilled'^ Until now the moie significant contiibution in 
test constuiction has come fiom the method of inbreeding of 
test items as' utilized by Toops in the constuiction of the 
Ohio Psychological Examination ” By means of this con- 
tinuous process of selection of the most valid and most stable 
items, the predictive validity of the Ohio test has at times sui- 
passed ,60 The piomise for furthei developments from this 
source is at piesent greatei than from the method of factor 
analysis The stimulus foi furthei advances by the method 
of inbieeding piobably will come from studying the contribii 
tions of the alternatives in a multiple choice item ^ But even 
with this contribution the piospects for the near fiitiiie aie not 

'^Stalnnkei, op eii See also Bernrenter, op ctl 

A Toops, ^*rhe Evolution of the Ohio State University Fiychological 
Test,*’ OA;i) Coilepe Asioatttioii Bullett?! JVc 1113, March 20, 1939, pp 2267-311 

F Kuder, The Consintctioti of Valid Test Items (Unpublished Disaer- 
Intioii, Ohio Slate Univeraitv, June, 1937) 
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voiy bright foi (i laige iiicicnsc in icliability and valiclitv 
□f tests 

The second coiusc, inci easing the leliability of the ciitciion 
measures, spoiJidjcally has been the topic of intense discubsioii 
in educational ciides As feii back as 1913 Staich appealed 
for moie stable giading standaids The majoi factois which 
he cited as the cause foi instability of maiks still aic applic- 
able today 

"(1) Diftcrcnccs among standaids of diffcient sdinols, (2) diHfci- 
ences among standards of different tcachcis, (3) dififcicnccs in the 
relative values placed by different tcaclicis upon vaiious elements in a 
paper, and (4) differences due to pure inability to distinguisli bet^^ecn 
closely allied degrees of meiit 

In the last decade a new type of emphasis in tlie giading 
pjocess has aiisen laigely thiougli the iiiflnence of Tylci® and 
the Piogicssivc Education Association evaluation woik The 
effoits of this gioiip have been diiectcd mainly towaid the 
claiification of tcacheis' aims and objectives and the opcia- 
tional deriiiition of these aims and objectives in Lei ms of 
obseivable bebavioi These developments have been diiccted 
chiefly at the secondaiy school level in connection with the 
Eight-Year Study. 

The widei application of these piinciples at the college 
level may offei some hope foi the impiovcment of picdiction 
It is assumed that this type of study will lead to a moie con- 
scious and a more stable evaluative pioccss which in turn 
should seive to make giacles moie leliabic, Some believe that 
gieater homogeneity in the objectives of giades also would 
result ftom these developments That is to say, many objec- 
tives piobably could be identical foi diffeient coiiiscs If a 
coie of common objectives could be isolated and evaluated 
in the same maimei in a whole senes of couises, then the 
difficulty of consti acting a moie efficient test batteiy would 
be reduced consideiably 

Sfifch, "Reliabilily and Diatribnlron of Giades/’ Sciefitt, XXXVHI 

(1913), 630 

®Ralph W Tyler, "Needed Research m riie Field of Tests And Exnmmii- 
iions," EditcaUoifai Research BnlleUn, XV (1936), 1SI-5S 
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The thiul approach is most likely, we believe, to bring 
abovit significant incieascs in the predictive efficiency of test 
batteiies and is one that piobably would encounter the most 
opposition fiom admlnistiatoi's and faculty If we assume that 
the piesent grade critciion of college success lacks adequate 
predictability and that this deficiency wan ants the substitu- 
tion of ii moic predictable cnterion, then is it not logical to 
seek such a criterion? The answer can be only m the affirma 
tivc At least a poition of out efforts must be directed at the 
possibilities of developing a moie predictable measure of 
academic achievement which at the same time will satisfy 
othei needs of the educational program 

But we also must consider the difficulties of such an under 
taking and be piepaied to surmount them Over and above 
the educational and statistical problems are the sociological 
pioblema which anse from the nature of oiu educational 
society This society has developed a rigid and inflcMblc 
attitude toward maiks which is likely to resist any but veiy 
stiong piessuies 

To dislodge the tiadition of marks, two foices must be 
overcome /list, the faculty, who feel that they have a vested 
mteiest in assigning grades, and second, paients, who, ‘indif- 
ferent at times to most phases of education, seldom neglect 
the report caid This tigid adhcicnce to marks has another 
deleterious effect upon the educational process Instead of 
diiectiiig their elloits toward masteiy of content, many stu- 
dents prepare for grades Originally designed to serve merely 
as a record that a student had taken a particnlai course and 
had acquired a certain degree of proficiency, grades too often 
have become the only goal for many students Foerstet has 
described the situation in pungent terms 

“Once a credit was earned, it was as safe as anything in the world, 
It would be deposited and indelibly recorded in the registrar's savings 
bank, while the substance of the course would be, if one wishes, happdy 
foigottcn Each course culminated in a final exaimnation* if one knew 
one's stuff then, one need never know it again In a subject like rcquiicd 

0 Billen, Provisiojts for Individual Diffennces, Marking and Promo- 
lion (U S Office of Education, Bulletin l!>32, No 17), p 4S9 
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English, a student deficient in ability might, with clfoit, get a passing 
grade, and then, without effort, pass into semi-iIliteracy , yet the icc- 
ord would show, to the day of doom, that he could icad and wiite 
passably 

All this menns that mstitutions of liighei leaining may 
have to abandon or modify the traditional marking system and 
"pioduce a new convention better than the old.”” It is thus 
seen that continued enslavement to traditional maiking systems 
not only interferes with the constuictioii of more effective 
selection instruments, but also vitiates some of the funda- 
mental objectives of highei education Theiefore, upon the 
shoulders of the educational administiator falls the lespon- 
sibility of re-examining the purposes of a marking system — a 
system that he eithei implicitly oi explicitly has set up as 
proper In this re-exainination he will be obliged to leave the 
way open for the substitution of another marking system 
which will provide optimal satisfaction of these purposes 

Oiir problem has come into sharp focus a new stand aid 
foi gauging achievement in college must be sought This 
standard must palpably be supcnoi to tcacheis' maiks and 
must icst on ccitain logical and statistical pillais Oui pie- 
vioiis discussion of the limitations upon piedictivc accutacy 
for college selection purposes has already indicated some 
of the desirable featuics" first, the measure should have 
leliability, second, it should be as homogeneous as possible 
both with respect to scale and to the nature of the factors 
included, third, it must have relevancy for the educational 
objectives to be measmed 

Since the final test of the piedictability of a ciitcrion will 
be empirical, we turn to the data that we have to piegeiU 
At this juncture the natiiie of the statistical evidence we have 
obtained forces us to particulari/.e in terms of the Iibeial aits 
college, and more specifically, the junior division The undei- 
lying principles, however, can readily be adapted to other 
college units Before proceeding with the analysis of the com- 

^^Noiman Foerstcr, The American Stafe Uuiverjity (Chipcl Hill, N C , 
University of North Carolina Press, 1937), p 97 

^^Ibid , p 146 
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pnjativc pi cclictabilily of the two types of cuteiia, a woid 
must be said about the objective of the junior division of the 
libeial aits college At the iisk of seeming impeitincntly pre- 
sumptuous 111 stating this objective in a woid, the authois sug- 
gest that the pnmaiy puiposc of the fiist two pi e-spec i all? a- 
tion yeais in the aits college is to pi o vide students with oppoi- 
Lunitics ioi cultiual gioAVth. Gcneially speaking, today's lib 
eial aits colleges by and laige diicct then effoits — sometimes 
futilely — towaid a cultiual goal Although in this context the 
word ciiltiiie is to be looked upon with the giavest suspicion, 
the trend in today's coie ciiiucula seems to be away fiom the 
hot-house vaiiety of cultiue for the elite and in the direction 
of the by-piodncts of the best in science and society for all 
Tn most cases, the Ameiican liheial aits college is bent upon 
pioviding students with a bioad undci standing of cultuie in 
all of its lamifications 

To be cultuicd, a man must be “moic than an ape-like 
cicatiuc posing undci the mask of hastily acquiied di awing 
loom nifinneis The student must acquiie dining his pic 
speciali/iUion yeais the individual qualities and competences 
which go into iich and satisfying living, and which give mean- 
ing to his expel lences as a membei of society This does not 
imply that a cultiual pattern iigiclly common to all is the goal 
of liberal education To di agoon widely diffei ent students into 
a legion of legimented automatons, each lesponding in the 
same way to the same situations, is obviously to be deplored in 
demociatic institutions As Eckert has phiased it “Not like 
minded, but ‘fiee’ individuals become the goal of Leaching 
Idlosyiiciatic behavioi lemains as an outstanding desideratum 
of liberal education 

At this point one of two approaches is immediately appar- 
ent foi evaluating these cultural objectives We may letieat 
to the traditional methods of evaluation — teacheis’ grades 
based on some esoteric combination of impiovised testing, 

J Wquletij The Lviergence of Huvian CuUure (New York Mocmillan 
Company, p B 

i^RiiIh E Eckeit, '‘Who Are the Cultured in Our Colleges?" Educational 
'Recofd, Januaiy, 1930, pp 133-35 
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dnzdmg intuitions, and the pcisistence of the student ni 
attending classes, oi we may tuin to piocediiics such as those 
embodied in ceitain umfoim testing piogiams 

Such a measure of cultuie could be assembled 'ivith the 
Coopeiative Geiieial Ciiltiuc test as a nucleus, Since 19'12) 
the content of the Sophomotc Cultine test has been considei- 
ablj^ expanded, and today the student urns ,i gamut o( tests 
horn mathematics to aesthetic appieciation bcfoic enteunghis 
junioi yeai Admittedly, the papei-and-pencil insliumcnt does 
not sample the whole lange of cnltinc, neithei does it diiectly 
tap the impojtant aieas of motivation, attitudes, and values 
It IS the only method yet devised, hovvevei, which has the fun- 
damental chaiacteiistJcs without which bcientific mefisiuement 
in education becomes a faice, a tiagedy, oi both Tf college 
administratois and faculties aie to decide whethei the Soph- 
omoie CiiltLUe test will satisfy then needs, they must weigh 
It upon scales which cany empliical as well as logical weights 
If It IS agieed that the pieclictability of a pioposecl ciitciion 
IS one chaiacteiistic peitinent to its adoption, the piedictability 
of the Sophoinoie Ciiltiuc test becomes a mattei of moment 

One might point out that it has a high leliability — cocITi*. 
cients in the 90‘s aie lepoited m the litciatuie — oi that it is 
constructed so as to give compaiable scoics, but the final pioof 
of Its supeiioi predictability must lest upon obtained coiiela- 
tioiis with piedictive tests The lemaindei of this papei is 
devoted to an exploiatoiy investigation of the predictability 
of the two ciiteria we have been discussing — teacheis* maika’^“ 
and the Sophomoie Culture battery, 

The usual technique was employed in assembling a batteiy 

^''The Cooperative General Cnllnre tesl may be procured fiom the Coop- 
erallye Teat Service, 15 Ainaterdam Avenue, New York The other Cooperative 
tests used in this aliuly may be obtained fiom the some souicc, 

i-0Teacher3' m'lrka weie tranamuted to two >car Jianor'poini ratios as 
Follows for each cicdit bom m which an A wns recoidetl, three honor poiiiLa 
were assigned, for each ciedit hour of B, two honoi points, foi ench ciedil lioiu 
of C, one honoi point, foi each cicdit hom of D, no hoaoi poiius, niul foi each 
credit hom of F (failing;) one honor point wns subtracted The honoi^poiiit intio 
was computed by dividing the total xiiiniber of liojioi pojnta enjiied by the 
total credit hours earned Ihe Sophomore Culture test was made up of the 
following tests m the Cooperative senes foi I93fi General Ciifliiie, English, 
General Science, Literary Acquaintance! 
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of tests foi the selection or rejection of applicants These 
tests wcie coirelnted first with teachers' marks and then with 
scores on the Sophomore Culture tests The battery of 
entrance tests which has the highest correlation with the 
ciiteiioii could thus be used to select future candidates for 
admission At the time of the students* entrance into the arts 
college, scoies on the following measiiics weie obtained, 

High school percentile rank 

Minnesota College Aptitude test (form AM) 

Minnesota College Aptitude test (fotm 1926) 
Cooperative English test (Pait I, form 1934) 

Cooperative Vocabulary test (Pait II of English test 
above) 

Coopeiative Contemporary Affairs test (form 1934) 

The group included in this study was composed of students 
who entered the arts college of the University of Minnesota 
as lieshmen in the fall of 1934, and who took the Sopho- 
more Cultiiie test in the spiing of 1936 in applying foi admis- 
sion Into the upper division The gioup was composed of 13S 
students, 56 men and 82 women Only students were mchidecl 
for whom the 1934 entuince test scores were available and foi 
whom high school percentiles were recorded, The group 
studied, though not closely lepresentative of enteiing fresh 
men, piobably was lepresentative of sophomoies applying foi 
entiancc to the senior division of the aits college Any liml 
tation in lepresentativeness, however, invalidates no compari- 
sons between different measiiies within this gioiip, 

TABLE 1 

CORRSLATrON9 DETWRBN TWO YEAR HOMOR-POlNT RATIOS AND iHDiViDUAr. MEASURES 

IN THE ERESUMAN TESTING HATTERY 



Total 

Men 

VFortien 

High achool peicentile mnk 

52 

57 

55 

CanlempoTuiy AEfaiia lest 

.50 

53 

44 

MiDnesoia College Aptitude test (1926) 

SO 

56 

+3 

CooperaUve English test 


SO 

41 

Minnesota College Aptitude lest (AM) 

+9 

49 

39 

Cooperative Vocabulary lest 

35 

40 

30 


Table t reveals the usual oidei of correlations between 
teachers’ marks and predictive tests The best single piedictor 
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of grades is the high school peicentile rank., demonstiating that 
—to a ceitain extent^ — -high school teacheis and college teachers 
are influenced by the same factois in assigning grades The 
highest con elation in the table is .57, between grades foi men 
and high school peicentile lanks The lowest coefficient, 10, 
IS between college gi ades foi women and the Vocabulary test 
The othei cocflicients fall between these two values. 

TABLE 2 

CORHELATIOHS DErVVEEW SOPHOMORE CUITURE TEST AND INDIVIDUM MPASURE3 IN 
TUB FRESHMAN TPSTINC OATTEHY 



Total 

Men 

Women 

Conternpoiary Affairs test 

81 

HI 

82 

Mionesota College Aptitude test (1926) 

77 

77 

.76 

Cooperative Vocabulary teat 

68 

63 

72 

Minnesota College Aptitiulc lest (AM) 

67 

68 

66 

Coapecat'vc English test , 

58 

62 

66 

High school peKcntilc lank 

.29 

•V3 

21 


Conti ast these con elation coeflicients with those in Tabic 
2 With the single exception of the high school peicentile 
rank, coiielations between the Sophomoie Culture test and 
the vaiious measiues lange fiom 82 to 58 The Conteni- 
poraiy Aftairs test has high piedictive value, as have the 
College Aptitude tests and the Vocabulaiy test The nature of 
the distribution of the English test scoics accounts for the 
lower coefhcient foi the total gioiip than foi eithei the men 
or the women It is especially notcwoithy that high school 
peicentile ranks have little piedictive value foi such a ciiterion 
In teims of foiccasting efficiencies foi the total gioup, the 
highest coefficient in Table 1 coiiesponds to 15 per cent, while 
the highest in Table 2 corresponds to 41 per cent 

The same trend appeals when the multiple con elation 
coefficients of selected batteries ate conipaied Table 3 leveals 
the order of coiielation between the two critena and two sets 
of entiance tests Batteiy A, composed of three measuies 
(high school peicentile rank, Minnesota College Aptitude test 

iTForecasiing efliciency compiUed by formula E = 100 (1— Vl— 1“) 
ivJnch gives a measure of the per cent of improvement over non-test pi ediction 
See J, P Guilford, Psyc/tometnc Methods (New York’ McGraw-Hill, 1936), 
P 363 
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— foLm 1926, and English test), corielates 64 with grades 
but 77 with theSophomoie Cultuie test. The coiiesponding m 
dexes of foiecasting eflicicncics ale 23 pei cent and 36 per cent 
When the Conteiiipoiaiy Alfaiis test is added to the thiee 
othei measuiet» (Batteiy B), the coudation with honoi-pomt 
ratio becomes 67, and with the Sophomoie Cultuie teat 86 
The coiiesponding foiccasting efliciencies aie 26 pei cent and 
50 pel cent bettei than iion-test picdicLion 

TABLE 3 

multiple correiahon coErncinNta detwben eattkries of selected entrance 

TESra AND TWO VEAR MONOR-POINT RATIO, AND SOmOMORE CULTURE TEST 

TwoYenr Sophomore 

IIonoi-FoitURnho Culture Test 

Total NIcn Women 'I otal Men Women 

Lntiancc llalteiy A* 6+ 67 6+ 77 711 77 

EntiniKC Bnltery Bt 67 63 .66 86 86 ,87 


*Enti nnee Baltery Acini'll school pciccntilc lankj Mtnncsotn College Apti 
nule lest (1926), and Cooperative English lest 

tEiurnnce Biuciy B— high school percentile I'liik, Minnesota College Apli- 
liiclc lest (1926), Coopci ilivc English test, nnd Contcmpoi nry Allans lest 

The dilieiences between these multiple coi relations for the 
two kinds of ciiteiia were tested foi significance The diffei- 
ences foi Batteiy A weie in the aiea of doubtful validity 
(P < 05 but > 02) , those foi Batteiy B were well beyond 
the boundary foi sigmbcaucc (P < 01) 

The significant lesults of this exploiatoiy study in the pie- 
diction of college success may be siimmaiized as follows (a) 
The substitution of the Sophomoie Cultuie test foi the con- 
ventional giading system as a ciiteiion of college achievement 
markedly inci eases the piedictive validities of the standaidized 
entiance tests and maikedly deci eases the piedictive validity 
of high school glades (b) The lowest validity coefficients 
weie obtained when high school percentile lanks weie coiie- 
kted with the Sophomore Culture batteiy. The highest zero 
oidei coefficients were obtained in coiielating the Sophomore 
Culture batteiy with the Contempoiaiy Affairs test Eckeit 

A Fiiher, SinUsfical Methods for Research JVorkers (7th ed , London 
Oliver and Bojel, 1939), p 209 


184 



NLW ClUiLR[A rOR 01!) 


lepoits sjmilai lindings. She concludes ; “Students most con- 
versant with the achievements and thoughts of the past, and 
most outstanding in the lealm of book-lcanimg, tend on the 
whole to be those inost aleit to the coiitempoiaiy scene 
(c) A combination of foiu entiance measmes i etui iicd validity 
coefficients with the Sophoinoic Cultuic test coiiesponding to 
$0 pel cent foiecastin^ efficiciity The Contempoiaiy Aftaii's 
test alone con elated highci with the Cultiuc Lest than did a 
combination of tbiee mensuies 

Inteipieting these lesuUs, the bophomoie Cultiuc Lest 
coil elates high with the other objective tests because of its 
close similaiity in objectivity of foim, its gieatei lelevancy 
and compithensiveness, and in the ovci lapping of the con- 
tent and ability measmes, and coil elates low with high-school 
giades because the lattci appiaise othei aieas besides those 
involved m the test sampling of achievement Giades in 
college, conveisely, con elate highei with giades in high 
school, and lowei with the sLandaidized tests because they 
measuie aicas outside of tested achievement but similai to 
those measuicd by high-school giades This inteipi ctation can 
be fiiithei suppoited anti extended since the con elation be- 
tween the Sophomoic CiiUiiie test and the two ycai honoi- 
point ratio was only modeiately high' 58 foi men and women 
combined, 64 foi men, and .51 foi women Scholastic giades 
and the Cultiuc test, even when they piesiimably sample the 
same areas of knowledge, ccitainly do not measuic all of the 
same aieas or abilities involved m academic achievement in 
college 

These lesults aie not without piccedent Foi example, 
Fiasiei and Heilman lepoited coirelations between the Thorn- 
dike Intelligence Examination and giades assigned subjectively 
and objectively The average coefficients were 45 and 60 
1 espeettvely Foi giades m Fiencli as assigned in the usual 
manner, Thaip found a coii elation of 47 with the Iowa Place- 

^“Eckert, op at,, p 135 

20G W P'r.'isier anfJ J D. Heilmnn, ''EjcpcrjmenU jii icachcf College 
AdminiBtration, lU Intelligence Tesla,” EdnctiHonal /IdtmmttfaUon and 
Superv\s\on, XIV (1928), 268-7B 
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ment test for foieigii language aptitude When an achieve- 
ment test was used, the coi relation jumped to 64 

From oin examination of the problem of piediction, we 
draiv the conclusion that a fiiiitful point of attack is through 
the substitution of a moie icliablc and theiefoie more predict 
able measure of achievement This papci has piesented data 
which definitely demonstrates that a pencil-and-papei evaliia 
tion instrument such as the Sophomore Cultuie test is more 
piedictable than the time-honoied giadc ciiteuon But it 
would be foolhaidy indeed for the authois to take the next 
step, that of advocating that this attiibute alone justified its 
substitution foi honoi-point latio This decision lies within 
the province of the educational administi atoi He must decide 
whether more accuiatc piediction — a snie qua non of all effi- 
cient admissions policies — plus the Ciiltuie test's degiee of 
lelevance is sufficient to outweigh those desiiablc qualities 
which may still be claimed foi the tiaditioiial maiking system 
In shoit, he must decide whethei this new ciiteuon is moic 
acceptable than the old 

A final word foi leseaich The Sophomore Cultiue test, 
in common with othei achievement tests, laigcly measuies 
recall of infoimation That infoimatioii is only one phase 
of education must be lecogni^ed Othei components of cul- 
tural giowth — attitudes, values, motivations, goals, and affec 
tive experience — must be mcasuied by othei msUuments It 
Is hoped that in the not-too-distant fiitine these impoitant out- 
comes of education can be appiaised with sufficient accuiacy 
so that we may know how well the Amciican college functions 
as the vehicle of culture 

B Tharp, "Sectioning Clastic*] in Romance Laugiiagcsj'' Mothm 
Lauguflt/e Journalj XII (1927), 95 114 

E CuTcton, "ETaluatiDTi or Guidance — A RepoA of the 1939 Sopho 
more Testing Progiam,” Journal of Experimental Ldncation, VIII (1940), 
308-40 
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A FACTOR ANALYSIS OF A NON-VERBAL 
REASONING TEST 

ROBL.R1 I. BLAKLY 
Social Security Boaid 

S OME time ago Dr Andiew W. Brown and the aiithoi 
constructed a “Noii-Veibal Reasoning Test” foi use at 
the high school level A pieliniinary lepoit of its constiuction 
IS being published by The Jonnial of Educational Psychology 
The present aiticlc concerns itself with the icsiiJts of a factoi 
analysis of the inteicoiiclations between the subtests rathei 
than with the actual standauli/ation of the test 

The test was constiuctcd with the idea that it should 
measuic in a non-veibal mannci the highci intellective pioc- 
esscsof compichensioji, mental alcitness, deductive leasoniiig, 
iiidnctive iCfisoning, and spatial lelations oi analysis The pii- 
mary purpose of this study is to isolate and identify any 
common factoi s piescnt and to compaie them with the ex- 
pected tactoia 

Other piobicms which may be consiclcied in the light of 
the factoi analysis aie (a) a comparison of the factorial 
composition of tests which aie variations of Thiirstone’s tests 
with the factoi lal composition, as deteimined by Tliurstone, 
of the tests he used, (b) a reconsideration of the perennial 
problem of the existence of a geneial factor of mental ability; 
(c) the comparison of the factoi s found in this gioup of 
tests with factors found In analyses of other tests, (d) a fur- 
ther examination of vaiious methods of ascertaining the num- 
bci of factors which should be taken out of a correlation 
matrix 
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All tests aie time-limit tests and weie intiodnced by foie- 
cxercises which weie explained by the examinei They weie 
piescrvted in the oidei listed 

1 Mnmkw^n page of pied ligiucs of little men The 
figures aie simple line diawings with vaiiations m the positions 
of arms and legs The pioblem is to diaw a iing aiound each 
manikin which is exactly like a model at the top of the page 
It was thought that this test might be satniated with the 
Peiceptiial Speed factoi The Speaiman-Biown collected 
ichabdity is 81 

2 Identical Patterns — 12 lows of patterns loimed by 
oveilapping geometrical foims The first pattern of each 
low is sepaiated fiom the othcis by a heavy vertical Ime 
The patteuis ave in 12 vaiiations each composed of two cnclcs 
and two light tiiangles The same size foims are used in 
each vaiiation, the diffeiciiccs being due to lelative positions 
of the components and whethei the foims aie solid oi dotted 
lines Each row contains one or moie patterns exactly like 
the fiist one in the low, and the pioblem is to place a maik 
ufidei each pattern which is exactly like the fiist one in its 
lespective low It was thought that this test would be a vana' 
tion of Thurstone's Identical Fonus test and consequently 
loaded with the Peiceptual Speed factoi The Speaimaii 
Blown coriected leliabihty is 98 

3 Fitting Pails — each item consists of a solid black 
geometrical foimj which has been cut into three paits, and 
foiii Outlined figiiies, one of which is the same size and shape 
as the black figure which was cut The problem is to indicate 
that one of the outline forms into which the solid black 
pieces could be made to lit exactly Disci iminatioii of both 
size and shape is involved foi each item It was thought that 
possibly the factoi Visualization or Space was involved in the 
solution of this test The Speaiman-Brown coriected leliabil- 
ity of the 12-item test is 47 

4 Opposite Sides — each item consists of three diawmgs 
identical in size and shape The pioblem is to select the 
drawing in each item which is a mirioi image of the othci 
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two drawings Each drawing is a litlle pennant the shape 
of a non-isosceles right tiiangle and may be rotated in any 
position, It was thought that possibly Space and Induction 
might be used m the solution of this test Theie is no ically 
parallel foim to this test although the idea was adopted fioin 
Thuistone’s Flags test The bpeaiman-Brown collected le- 

liability IS 88 

5 Code — a code consisting of eight boxes divided in 
half JS placed at the top of the test Each box has a unique 
group of squares and ciicles in the top half and an unusual 
group of tiiangles in the bottom half Below the "code’^ arc 
five rows of the little, boxes, some exactly like the boxes in 
the code and some with inconcct paiiing of the symbols 
The problem is to place a Imc iindei each box which is dideient 
fiom the code It was thought that the test might contain the 
Peiceptiial factoi The Speaiman-Biown collected leliabihty 
IS 96 

6, Cncle Gtoni)tng — each item consists of foui boxes 
containing little gioups of ciicles The giouping vanes fiom 
box to box One cncle m each of the fust thiee boxes is 
blackened accoi cling to a system The pioblem is to discovei 
that system and apply it in blackening a cncle in the fmuth 
box It was thought that possibly Induction would be involved 
in solving this test The Speaiman-Biown collected leliability 
foi the l2-item test is 98 

7 Foim Sene '; — this test is the usual senes type with 
only thiee meaningless foims used In combination One figure 
in each low is omitted and a blank inseited. The pioblem is 
to indicate which form belongs in the blank It was thought 
that Deductive Reasoning oi Inductive Reasoning would be 
involved m the solution of this test The corrected Spearman- 
Biown leliability of the 22-item test is 86 

8 Cttcle Reaso?fntg — a vaiiatlon of the Marks test used 
by Thurstone as a measiue of Inductive Reasoning There 
are five lows of gioups of circles and dashes The grouping 
changes fiom low to row One circle in each of the first 
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four rows is blfickciied accoi cling to a lule, The problem is 
to find the rule and apply it in blackening a ciicle m the fifth 
row It was assumed that this test would contain Induction 
The collected reliability la 94 

9 Fofni. RehittoHS — this test is a paiallel foim of Thiir- 
stone's Paftetii Auoiogtes test The problem is to indicate 
one of five choices which beais the same i elation to the thud 
figuie as the second beats to the fust, Inductive Reasoning 
or Deductive Reasoning was assumed to be necessary for the 
solution of this test. The coriected leliability is 97 

10. Foun Reasoumg — at the top of the test is a table 
showing how any two of seven foims could be combined to 
equal another one of the seven Each item consists of thiee 
of the forms in a row The task is to combine the first two 
foims accoiding to the table and then combine the resulting 
foim with the thiid to equal another foim, the final lesiilt 
to be indicated by imdei lining one of five choices It was 
thought that possibly Deductive Reasoning would be used 
to solve these problems The Speaiman-Brown collected 
reliability for the 12-item test is ,98 

The Subjecls 

The subjects weie 286 high school pupils from a school 
in a suburb of Chicago All tests were given by two experi- 
enced examiners in a well-lighted room All tests were admin- 
istered in one 40-minute period Eighty pei cent of the whole 
gioiip was between IS and 18 yeais of age The mean Otis 
IQ was 114 About 54 pei cent of the group were boys 
No sex difference was found for combined scores on the whole 
test No grade difference was statistically significant The 
correlation of total test score with chronological age was — 1 1 
for the age langc of this group 

The Facto? Ana} pis 

The table of intercorrelations (Table 1) was computed 
with the aid of Coinpuhng Diagrams for the Tetvachoric 
Coi relation Coefficie}?t (2) Correlations obtained m this 
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nifinnei a?e consideied by Thmstonc (6, p 58) to he applicfi- 
blc to factor analysis In effect the scoies are noimalizcd in 
the process of correlation 

The factors (Table 2) weie extracted by the Tluiistone 
centroid methods Heic the problem of the numbei of factois 


'lAJlLE i 

IW rrjICORRHI WHINS or 'JJSJS 


Variable 

1 


2 

3 

4 

S 

6 7 

B 5 

1 10 

Manikin 



24 

27 

24 

38 

19 13 

.19 22 19 

Identical Patterns 

2+ 



08 

17 

22 

46 (6 

15 33 24 

Fitting; Parts 

27 


o« 


17 

22 

20 13 

10 20 22 

Opposite Sides 

2+ 


17 

17 


15 

25 38 

32 39 31 

Codfr 

38 


22 

22 

IS 


26 .22 

25 3 

5 38 

Circle Grouping 

19 


46 

20 

25 

26 

48 

SO 53 49 

Fwm Senes 

13 


16 

13 

38 

22 

4B 

35 52 54 

Circle Reasoning 

19 


IS 

10 

32 

25 

SO 35 

55 38 

Form Relntions 

22 


33 

20 

39 

35 

53 52 

55 

40 

Form Reasoning 

19 


24 

22 

31 

38 

49 54 

38 40 





TABLE 2 







CFNmOiP MATRIX 












Factors 



Variable 

Cor: 

Ic 

No 

1 


11 

in 

IV 

V 

Maniliin 


1 


408 


+35 

“183 

— 083 

— 069 

Identical Patterns 


2 


.4S2 

— 

141 

27+ 

263 

— 200 

Fitting Parts 


3 


335 

— 

,212 

— 101 

— 140 

087 

Opposite Sides 


4 


499 


loo 

— 163 

— 112 

— 242 

Code 


5 


SQ6 

— 

297 

— 055 

— 079 

130 

Circle Grouping 


6 


701 


US 

296 

272 

109 

Form Senes 


7 


622 


377 

110 

— 274 

— 117 

Circle Reasoning 


B 


602 


2B1 

— 252 

238 

205 

Form Relations 

h 

9 


728 


181 

— 154 

177 

— 093 

Form Reasoning 

10 


665 


119 

,166 

— 251 

.239 


appealed Two methods of deteimining the number of factors 
had been tried by the anthoi (1) previously with some degiee 
of success One of these, Tuckei's empirical ciiteiion, gave 
negative lesiilts in the picsent case The other, Coombs* 
ciitei ion (3) postulates that in a 10-variable problem, the 
last factor of value will leave a table of residuals which, when 
signs aie changed, will contain more than 31 negative entries 
with a standard error of five Tabic 3 shows the application 
of Coombs' cnteiion to this analysis’ 
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Fnctai 

1 

2 

3 

4 

5 


TABLE 3 
COOMDS^ CRITERION 


Negatives 

24 

33 

. 2 + 

2i 

35 


It was obvious Tiom the iiumbei of i datively laige 
lesiduals lemaiiiing in the table aftei the second factor was 
e\ti acted that theie were moie than two factois in the table 
This was borne out m the subsequent analysis, which was 
earned to five factois The indication that the fifth factoi 
was the last one of value seems to have been veiified m the 
analysis The standaid deviation of the fifth factoi lesiduaU 
bcfoie sign change is 028, which is consideiably smallei than 
the standaid eiioi of a 7 eio con elation foi a population 
of 286 

Foi the lotation of factois in oidei to seciue bounding 
hypei planes, Thmslone's method of lengthened vectois was 
used (4). The ciiteiia of maximizing the mimbei of zeios 
and lotating to a postulated positive manifold weie the deter- 
mineis foi dhection of lotation Seven lotations weie neces- 
saiy and a ‘'clean-up" lotation with actual length vectois was 
made The lotatcd factoiial matiix is given in Table 4 
The lotational matrix of diiectioii cosines is given in Table S 
The inteicoiielations between the lotated factois are pie- 
seiited m Table 6 


TABLE 4 

ROTATED rACrORlAl MATRIX (lA) 


Vaiiable 

Manikin 

IdeiUicnl Falterna 
Fitting Pans 
Opposite Sides 
Code 

Circle Giouping 
Foim Series 
Circle Reasoning 
Foino Relations 
Foim Reasoning 


Code No A B 


1 

592 

075 

2 

092 

547 

3 

345 

— 041 

4 

132 

028 

5 

440 

067 

6 

— 076 

436 

7 

— U1 

040 

8 

— DIG 

— 046 

9 

076 

162 

10 

080 

021 


192 


Futtor 


C 

D 

E 

004 

192 

05+ 

— 016 

239 

014 

— 009 

265 

00 S 

160 

313 

408 

00+ 

394 

^062 

161 

641 

— 073 

016 

639 

453 

561 

518 

026 

415 

507 

2+4 

— 053 

766 

071 
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TABLE S 

TRANSrORMAllON MATRIX (A) 


Reference Vector 


Centfoul Axis 

A 

B 

C 

D 

E 

^ I 

287 

247 

207 

BOl 

203 

II 

— 859 - 

•2+3 

361 

2+8 

379 

III , 

— 380 

671 

— 690 

23+ 

— 22+ 

IV 

— 18+ 

521 

582 

— 2S7 

— +36 

V 

033 — 

•399 

111 

+21 

— 759 


lABLE 6 






CORRClAriONS Ul-rWPEN NORMAI5TO nil 

PIANI'S (A' A) 





Plane 



FidllC 

A 

B 

C 

D 

E 

A 

1 000 





B 

— 0B+ 

1 001 




C 

— 092 

— 2+1 

1 000 



D 

— Oil 

— 007 

— 009 

1001 


E 

— 127 

— 117 

— 005 

— 003 

1001 


Even a ciubory glance at the lotatcci matiix will show 
that the factoiial composition of the tests is not so simple 
as had been hoped foi, 

Factoi has three vaiiables with significant piojcctions 
and all the otheis aic essentially zcio These aie 

1 Manikin * . 58 

3 FiUiug Patios , , 3^ 

5. Code . . 44 

Eithei one of two Inteipietations could be placed on this 
factor It might be considered to be Space as has been de- 
scribed byTliuistone (6), the aiithoi (1), and otheis Undei 
this Intel pretation it would seem that the giasping of spatial 
relations of the arms and legs of the maiukins was of moie 
impoitance than the quick perception of small diffeietices It 
Avould appeal also, that the quick compaiison of the code 
with the stimuli in the Code test was not so impoitant in 
solving the pioblem as the giasping of the relationship be- 
tween the two halves of the individual elements 

The othei intcrpietation which could be placed on this 
factoi IS that it is Peiceptual Speed, oi lather mental aleit- 
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ness, as distinguished fiom Perceptual Disci imination Under 
this inteipietation the ability would involve the quick change 
of icsponse fiom item to Item with only the simplest dis- 
crimination necessaiy Thiii stone’s factor ‘‘9” in his study 
of Hyde Park High School m Chicago seems to have some 
of the cliaiactenstics of factoi *'A” (5) In this case, the 
test Scattered had the highest loading The Ma?iikiu 
test has the simplest disci imination level and the Fitting Pails 
test the most complex of those listed The author prefers 
this latter inteipretation 

Factoi '’B” has two tests which have significant loadings* 

2 Identical Patterns , 55 

6 Citcle Gioupnig , 44 

It seems obvious that this (actoi coiicsponds to Thur- 
ston e's (6) Peiccptual Speed factoi, bvit we shall call it 
Peiceptual Disciimination to distinguish it from factor "A" 
The difference here is that the emphasis is on analytic per- 
ception in which a fine discrimination must be made rather 
than on speedy icsponse to a simple stimulus Speed is of 
impoitance, but in the subjects used the diffeiences in the 
mental process of peiccptual discrimination will contribute 
moie to peifoimance vaiiance than will simple speed 

At fust glance it seems SLUpusing that Ciule Giouping 
is high on this factoi However, a careful subjective analysis 
of the test will indicate that the problems involved are more 
those of peiceptual discrimination than of induction The 
figuies are complex but the lules to be brought out aie simple 
Foi example, one of the items has the middle dot blackened 
in a group of three, which is appaient even at a glance, so 
that the problem resolves into finding the coirect gioup m 
the lesponse squaie This takes a disci immatoiy ability 
evidently slightly below that requiied for Identical Patterns 

Factor “C” has two variables with significant pi ejections* 

8 Clide Reasoning , 56 

9 Fotni Relations , .... 42 
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Both of these tests are vfiriations of tests used by Thui- 
stone in his studies of the primary mental abilities and have 
been interpreted to contain Induction, oi Inductive Reasoning 
This interpietation is suitable in the present case The ap- 
parent paiadox that test 8 contains Induction ’while test 6, 
in which a supposedly similar function is involved, does not 
may be resolved when an inspection is made of the tests them' 
selves The primary pioblcm in test 8 is to find a rule by 
which the problem may be solved while m test 6 the main 
problem, as has been said befoie, is to find the response gioup 
rather than the rule 

Factor ''D” IS an orthogonal factor which was set up by 
making its normal perpendicular to the normals of all the 
other planes This was necessary as one dimension of the 
five-dimensional system could not be identified by a bounding 
hypci plane because of lack of variables with zero projections 
in that dimension It is the same type of problem as was 
encoiinteicd by the authoi in a foimei study (1) 

All the variables have projections on this factoi which 
arc probably significant The relative amount of projection 
seems to incieasc with the grcatci complexity of the mental 
function involved The tenth test, Fo)vi Reasoning, which 
involves the synthesis of geometucal figuies accoiding to estab- 
lished rules (not unlike .iiithmetic), has much the highest 
satuiation of the factor, 

The obvious comment, and one that must be reckoned 
with, IS that this factor repiesents “general intelligence,” or 
Speaiman’s factor “g ” As has been said before, there is 
nothing in the Thurstone method of analysis which denies 
that such a general factor exists or implies that it would not 
show up if present However, in legard to the nature of the 
present factor, there can be little doubt that it is “general” 
for this battery of tests and is not an effect of maturation oi 
lack of differentiation of ability due to the youth of the sub- 
jects, What it IS called — cofiipreheuston, understanding, 
mental efficiency, or intelhgunce — is beside the point Due to 
the popular misconceptions and scientific vagueness of the last 
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tcim, it piobably would be bettei to adopt some other name 

It should be iindei stood that the aiithoi is of the opinion 
that the above-mentioned effect of an augmented general factor 
due to lack of matuiation ts applicable to situations In which 
the subjects aie immatiiie, but that such a factoi does not 
account foi appieciahle distoition in the piesent case It is not 
denied that such a geneial factoi is piesent in tests given to 
childien, but it seems piobable that the geneial factoi, if it 
exists in such a case, is unduly emphasized by the matuiation 
cuives of the abilities 

Anothei inteipietation which might be placed on factor 
“D" IS that It IS Deductive Reasoning, which in each test 
leqiiiies that the subject must base his conclusions oi responses 
on ceitain facts which aie picsented in the test item How- 
cvci, this IS piobably .uiothci aspect of the foiegoing 
discussion 

Factoi "E” has significant loadings foi two tests and a 
possibly significant loading for a thud 

4 Opposite Sides , 41 

7 Fonn Seties . . , .45 

9 Fo) ?n RelaHons 24 

This factoi appaiently coiiesponds with none of the fac- 
toi s picvionsly identified by Tluustone and his associates 
Howevei, it may possibly icpresent Deductive Reasoning as 
“senes” tests have been found by Thurstone (5) to contain 
a component of Deductive Reasoning The same is true of 
the foim lelations type of test The relationship of the 
Opposite Sides test to such an inteipietation is not immediately 
appaient Assuming that one might consider two figures m 
each Item of the Opposite Sides test as facts to be compared 
and fiom which a conclusion might be diawn concerning the 
thud figiiie, 1 e , whether it is diffeient from the fiist two oi 
like one of them, then it might be thought to involve Deduc- 
tion In the Foim Seites test the symbols presented are facts 
fiom which a conclusion must be drawn conceining the missing 
figuic The conclusion is definitely limited to three altev natives 
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each of which might be tiled m tuin In the Fotm Relations 
test the problem might be appi cached by tiyiiig to find the 
rule mvolved, which would be Induction, oi by substituting the 
possible aitsweis one at a time and testing the lesulting equa- 
tion This lattei piocess might be consideied to be Deductive 
Reasoning and insofai as it weie used would cause the test to 
show a loading on the Deduction factoi 

No definite conclusion can be made as to the identity 
of Factoi "E,” but tentatively it may be called Deductive 
Reasoning 

Despite the fact that the factoi lal composition of some 
of the tests vanes somewhat fiom what was oiigmally sup- 
posed, it seems that the tests, as a gioiip, do measiue some 
of the highei mental piocesses of leasonlng Fiom amount 
of piojectioii on the geiieial factoi, it would seem that the 
tests satuiated with Peiceptual Speed aie the pooiest mea sines 
ol the highci intellective pioccsses It would appeal that Lest 
juimbei 9, Foim Reiaiionsy which has significant piojections 
on thiee factois, is piobably the best geneial test of all the 
leasoning piocesses Test 10, Foim Reasomn^j is the best 
test of the geneial factoi which might be consideied to be 
synonymous with compichcnsion oi mental efficiency oi intel- 
ligence The test. Identical Pal lei ns, seems to be satin nted 
with the factor Peiceptual Disci imination, which is intei- 
pieted quite similaily to Tluu stone's factoi ol Peiceptual 
Speed, and is consistent with Thurstone’s (5) test of Identical 
Foims, which is paiallel m piocess The test, Cticle Reason- 
ing, a vaiiation of Thuistone*s (5) Maiks test, is similai in 
factoi lal composition to the lattei The Foim Relations test 
seems to have a heteiogeneous factoi lal makeup, as was also 
found by Thui stone (6) 

The factors identified seem to be consistent with those 
identified by Thuistone (6,5) except foi the geneial factoi 
It IS necessary to investigate these tests m a laiger batteiy 
befoie an inteipretation can be adequately applied to the gen- 
eial factoi This factoi has some chaiacteiistics similar to 
those found by the authoi (1) in factoi m a “Reanalysis 

197 



ET^UCAllONAT, A>JD PSYCHOLOGICAL MEASUREMENT 


of {I Test of the Thcoiy of Two Factors The factor Per- 
ceptual Speed also seems similar to the factor "C” m the 
lattei study 

The factors have been found to be practically iincorielatedy 
the highest coi relation, that between factors “B" and “C>" 
being only 14 degrees off orthogonality Thts is probably 
within chance variation and no significance is attached to it, 
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California Capacity Questiomiau e, by Elizabeth T Sullivan, 
Willis W Clark, and Ernest W Tiegs 1941 For high 
school and college students, and adults Time, 30 minutes 
Foims A and B, 75^ pei 25 , 25^ pei specimen set Pub- 
lished by the California Test Buieau, 3636 Beveily Boule- 
vard, Los Angeles, California 


Cahfoima Test of PeuonalUy, by Louis P Thoipc, Willis 
W Claik, and Ernest W Tiegs 1940 One foim each 
foi piimaiy, elemental y, mtei mediate, secondaiy, and 
adult levels Time, about 45 minutes foi each senes 
Piimary senes foi giades 1-3, elementary seiies for giadcs 
4-9, intermediate senes for giades 7-10; secondaiy senes 
foi giades 9-14, adult senes, $1 00 pci 25 of each senes, 
25^ pel specimen set of each senes Published bv the Cali- 
foinia Test Biiieau, 3636 Beverly Boulevaid, Los Ange- 
les, California 


Coopei alive Comimimty Ajfaiis Test, by Roy A Price and 
Robert F Steadman 1941 Time, 30 minutes Foirn R, 
$3 50 per 100, 25^ per specimen set Published by the 
Cooperative Test Seivice, 15 Amsterdam Avenue, New 
York, New York 


Coopet alive Liteiaiy Gompi ehension and Apprectalion Test, 
by Hyman Elgeiman, Mary Willis, and Fiedenck B 
Davis 1941 Foi iippei high school and college classes 
Time, 40 minutes Form R, $4 50 per 100, 25^ per speci- 

♦Publishera and authois of new teats are requested to lend copies to The 
Editor^ Educaffoiial and Psyct/oJo^ical Measure uient, Dov 7^6, Alexandria, Va 
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men set Published by the Cooperative Test Seivice, 15 
Amsteidam Avenue, New York, New York 


CoQpe) alive Science Test, by John G Zimmerman and Rich- 
aid E Watson 1941 For grades 7, 8, and 9 Time, 
80 minutes Foim R; $5 50 per 100; 25(S pei specimen 
set, Pvibhshed by the Coopevative Test Service, 15 Am- 
steidam Avenue, New Yoik, New York 


Caopei alive Social Studies Test, by Agatha Townsend and 
Mary WiJlis 1941 Foi giades 7, 8, and 9 Time, 80 
minutes Form R, $5 50 pei 100, 25 {5 pei specimen set 
Published by the Coopeiative Test Service, 15 Amsterdam 
Avenue, New York, New York 


Dunlap Academic Ptefeieuce Blank, by Jack W Dunlap 
1940 Foi giades 7, 8, and 9, Forms A and B, 90^5 per 
25, 20^^ pel specimen set Published by the Woild Book 
Company, Yonkers, New York 


Eames Eye Test, by Thomas H Eames 1940 $3 50 foi 
examinees kit, 65 pej 25 individual recoid cards Pub- 
lished by the Woild Book Company, Yonkers, New York 


Examination foi the Measui cment of the Efficiency of Mental 
Fwiitioning, by Harriet Babcock and Lydia Levy 1940 
One form, set of test materials, $11 20, recoid blanks, 
$2 30 per 25, $6 90 per 100 Published by C H Stoelting 
Company, 424 NoUh Homan Avenue, Chicago, Illinois 


Foinih Glade Geogiaphy Test, by Zoe A Thialls, George 
Millei, and Maiguente Uttley 1940, For use at the 
end of the fourth giade Time, 35 rrunutes One form, 
8(1 per test, 4^ pci manuaj, 20^ pei scoiing stencil Pub 
lished by McKnight and McKnight, Bloomington, Illinois 
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Hills Economics Test, by Johti R Hills 1940 Foi high 
school and college students Time, 40 mrniites One foim, 
S0{^ per 25, 15(5 pei specimen set Published by Biueau 
of Educational Measiu cments, Kansas State Teacheis 
College, Empoua, Kansas 


Kansas Vocabulmy Test, by H. E SchiainmeJ, 0 M. Ras- 
mussen, Anna Huebeit, and D J Tate 1940 Foi grades 
4 to 8 Foims A and B , 40^ per 2S , 15^ pei specimen set 
Published by Biueau of Educational Measuiements, Kan^ 
sas State Teachers College, Empoija, Kansas 


Kiikpatnck Cheimstiy Test, by Einest Klikpatiick 1940 
For high school students Time, 40 minutes One foim, 
60^ pel 25, I5ti pci specimen set, Published by Biueau 
of Educational Measurements, Kansas State Teacheis 
College, Empoua, Kansas 


Kmss Woild Hisioiy Test, by F Roscoe Kniss 1940 Foi 
high school students Time, 50 minutes Foims A and B, 
$1 30 pel 25c, 20{* per specimen set Published by the 
World Boole Company, Yonkers, New Yoik 


Mechanical Compt ehenston Test, by Geoige K Bennett 
1940 For male high school students and adults Time, 
about 25 minutes One form, $2 50 pei 25 booklets and 
answer sheets, 2St pei specimen set Published by the 
Psychological Coiporation, 522 Fifth Avenue, New Yoik, 
New York 


Minnesota Pei sonahty Scale, by John G Dailey and Walter 
J McNamara 1941 Foi upper high school and college 
students Time, about 45 minutes Separate question 
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booklets foi men find women, answci sheet can be used 
\vith cithei question booklet, scorable by International 
Test Scoring Machine; $1 50 pci 25 question booklets, 
pei 25 answei sheets, 35^1 pei specimen set Published 
by the Psychological Coipoiation, 522 Fifth Avenue, New 
Yoik, New York 


Moidy-Schi^immel /iineuciiu Gaventment Test, by F E, 
Moldy and H E Schiammel 1940 Foi high school and 
college students Time, 40 minutes One foim, 50<} per 
25, 15t5 per specimen set Published by Buieau of Edu- 
cational Measulements, Kansas State Teacheis College, 
Empoiia, Kansas 


Mo'Kiy-Schiaiiim^i Co7t5i!iHiion Test, by F E Mordy and 
PI E Schiammel 1940, Foi high school and college 
students Time, 40 minutes One form, SCI^S pci 25, 15^ 
pel specimen set Published by the Bureau of Educational 
Mcasuicmeiits, Kansas State Peacheis College, Emporia, 
Kansas 


Peabody Ltbiaiy Infotmatwn Test, by Loins Shores and Jo- 
seph E Moore 1940 One form each for college, high 
school, and eJemeiitaiy school levels Time, 30 minutes 
College level one foim, *^1 00 per 25 High school level 
one form , 75() pei 25 Elementary school level one form, 
60^! per 25, 20^ pei specimen set Published by the Edu- 
cational Test Buieau, 720 Washington Avenue, S,E , 
Minneapolis, Minnesota. 


Tjigonometjy Test, by 0 M Rasmussen and 0 
J Peterson 1940 Foi high school and college students 
Time, 40 minutes, One form, 50^! pei 25; per speci- 
men set Published by Bureau of Educational Measure- 
ments, Kansas State Teachers College, Emporia, Kansas 
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Staufoid AchtevCfnent Test, by Tiunian L KeJlcy, LewJS M 
Teiman, and Giles M Riich 1941 Foims D and E for 
each of piimaiy, mteimediate, and advanced levels fiom 
grades 2 to 9 Piimaiy Batteiy, foi giades 2 and 3 
time, 50 minutes, $1 10 pei 25, 20(i per specimen set 
Inteimediate Battery — Complete, foi giades 4 to 6 time, 
150 minutes, $2 00 pei 25, 40fi pei specimen set Ad- 
vanced Batteiy — Complete, toi giades 7 to 9 time, 150 
minutes, $2 00 pei 25, 40^ pei specimen set Published 
by the Woild Book Company, Yonkers, New Yoik 


Tate Economic Geogiaphy Test, by D J Tate and G A 
Biizzaid 1940 Foi high school and college students 
Time, 50 minutes Forms A and B, 50jl per 25, 15^5 per 
specimen set Published by Bureau of Educational Meas- 
urements, Kansas State Tcacheis College, Empoiia, 
Kansas 


Tiuslei^Jineti Health Knowledge Test, by V T Tiusler, 
C E Ainett, Jr , and H E Schrammel 1940 For 
grades 9 to 12 and college Time, 50 minutes Foims A 
and B, 50<5 per 25, 15^ pei specimen set Published by 
Bureau of Educational Measurements, Kansas State 
Teachers College, Emporia, Kansas 


Turse Shorthand Aptitude Test, hy ¥d.\x\l^ Ilwvbc 1940 For 
use with high school students before enrolling in short- 
hand courses Time, 45 minutes One form, $1 30 per 25, 
10^ pel specimen set Published by the World Book Com- 
pany, Yonkeis, New York 


Vocational Inventory, by Curtis G Gentry 1940, For high 
school and college students, and adults Time, about 150 
minutes One form, 15fli for vocational inventory, mdt- 
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vuUial analysis lepoit, and individual scoie tabulation 
sheet, 2U pei sample set Published by the Educational 
Test Biueaii, 720 Washington Avenue, S E y Minneapolis, 
Minnesota 
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Adkins, Dorothy C and Ktrdei, G Frederic “The Relation 
of Piimary Mental Abilities to Activity Piefeiences" 
Psychoinetiika^ V (1940), 2Sl‘-62 

The relations of abilities, as measuied by Thut stone's 
Tests for Piimary Mental Abilities, to activity piefeiences, as 
measured by Kiider's Preference Record, are investigated for 
a population of 512 university freshmen Ability piofiles for 
contrasted groups on each preference scale leveal relatively 
slight oveiJapping bfetween the two sets of measures, although 
the apparent tiends are leasonable, The Pearson inteicoi- 
relatioii coefficients of all pairs of measures involved were 
determined Implications of the findings in relation to theory 
and to educational and vocational guidance aie indicated 
(Courtesy Psychovietnka ) 


Allison, G and Bainett, A “Freshman Psychological Exam- 
ination Scoies as Related to Size of Pligh Schools “ Joitt- 
nd of Applied Psychology, XXIV (1940), 651-52 

Quantitative and linguistic scores of 1,083 college fiesh- 
men on the 1938 edition of the A,C E Test weie analyzed 
with lefeience to the size of the high schools fiom which 
they g/aduated For thiee size-groups, statistically significant 
differences in means were found in five of six comparisons 
Means tend to inciease with eniollment but theie is much 
oveilapplng PF A Vaivel 


Andeison, H A and Traxlei, A E “The Reliability of the 
Reading of an English Essay Test," Part II School 
Review, XLVIII (1940), 521-30 


♦Edited by Professor Fojrest A Kingsbury 
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Factual notes wcie piepaied on two themes “The Dia- 
coveiy ol Gold in California^’ (Foim A), and “The Pony 
Expicss” (Foim B) , A gioup of 281 pupils in the Umveisity 
High School of the Univcisity ol Chicago were given the 
two foims at one yeai's inteival with insti actions to expand 
the mateiial into a two-houi essay The essays were giaded 
on a sixty-point scale with the following weights foi the sep- 
arate factois completeness (6), spelling (6), punctuation 
(6), language eiiois (6), coherence between mam divisions 
(10), orgam?ation of paiagiaphs (10), and oiganization of 
essay sentences (10) On leieading 70 essays of each foim 
the glades of a skilled leadei showed coiielations of 893 ± 
016 and 937 ± 010 foi the two foims, two leadeis, on 
fust scoimg of 25 papeis, showed coiielations of 859 ± 035 
and 898 ±. 026 foi the two foims Foi individual factors, 
no con elation was below 80 Giowth in language ability may 
be indicated by an aveiage gam of 3 3 points fiom Fmm A 
to Foim B foi 281 pupils The lesults aie not deemed con- 
clusive but only suggestive of the desii ability of expeiimenta- 
tioii with essay-test procediiies J E ICailm 

Babitz, Milton and Keys, Noel “A Method for Appioximat- 
mg the Aveiage Intei-Coiielation Coefficient by Corre- 
lating the Paits with the Sum of the Parts” Psycho- 
melnkaY (1940), 283-88 

It is noted that the aveiage mter-item cou elation, which 
lepiesents the mteinal consistency of a test, yields a unique 
estimate of test lehabihty A close approximation to this 
aveiage is given by a foimnla which leqiures the con elation 
of each item with the total score and the standard deviation 
of each item The foimula is especially useful m those in- 
stances wheie the numbci of items is small and wheie the 
variation m item sigmas should not be neglected (Couitesy 
Psycho in eut-ka ) 

Benton, A L and Periy, J D “A Study of the Piedictive 
Value of the Stanford Scientific Aptitude Test (Zyve) 
Jomnal of Psychology, X (1940), 309-12 
Scores on the Stanford Scientific Aptitude Test and the 
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ace Psychological Examination (1934-35) togethei with 
course giades foi 43 students ovei a peiiod of thiec to foiii 
yeais weie used in an investigation of the piedictive value of 
the Aptitude Test The aveiagc scoie on the ACE was 
appioximately one sigma above the mean foi the 1935 fiesh- 
men Coi relations of couise giades foi scientific and non- 
scientific couises with the Aptitude Test and the A.C E Test 
weie about + 35, there being no significant diffeience between 
the sets of coiielations The coefficients of coiielation of 
the 11 siibtests and aveiage giades in all college couises “were 
all quite low “ The authois suggest “that the test has a cei- 
tain limited value m piognosticatmg the scholastic achievement 
of fieshman and sophomore students “ Harold Bethtoldl 

Buios, Oscai Kiisen, Editoi The Nineteen Foity Mental 
Measui emenls ‘Yeaihook Highland Park, N J The 
Mental Measuicments Yeaibook, pp 674 -h xxxiii 1941 

The first pait of the Yeaibook contains leviews of new 
tests as well as of selected oldei tests Theie aie 524 tests 
listed Most of these aie reviewed by two oi three reviewers 
The second pait lists 368 books and pamphlets in the measure- 
ment field and excel pts fiom icviews of them which have 
been published in various journals 

Cast, B M D “The Efficiency of Difteient Methods of 
Marking English Composition ” Pait II BuiisJi Journal 
of Educational Psychology, X (1940), 49-60 
Forty English compositions weie marked by 12 examineis 
by foul different methods (1) the examiner's own habitual 
method, (2) the method of geneial impression; (3) Bait’s 
analytic method (allotting separate marks for specified points 
or qualities) , (4) Hai tog’s achievement method The P-tech- 
nique (correlation of pcisons) was combined with Burt’s 
summation method foi a factoiial analysis of the coiielations 
between examineis, this lesulted in (a) a geneial factoi (lep- 
lesenting the best appi oximation to the "tine marks”) account- 
ing loi 50 pel cent of the vaiiance; (b) a dichotomous factor 
of examineis maiking better by analytic methods oi by intu- 
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itive 01 impiessionistjc methods Tlie methods of maiking foi 
gencial use are hcie found to be in oidei of piefeience the 
"‘analytic” method, the method of geneial impiession, the 
examinei’s habitual method, and Hartog’s achievement 
method / E Kaihn 

Daniel, C “Statistically Significant Dilfeiences in Obseived 
Pci Cents” Jomunl of Applied Psycholoffy, XXIV 
(1940), 826-30, 

A table gives the amount by which a pei cent A obseived 
in one sample must exceed a pel cent B observed in another 
sample of the same size to be significant at the 0 05 level 
It IS presented for dilfeient values of B and foi samples from 
20 to 1,000 Vaiioiis conditions aie stated and the meaning 
and use of the table discussed PF A Varvel 

Davis, F B “The Intcipietatioii of I Q 's Derived fiom the 
1937 Revision of the Stanfoid-Binct Scales” Journal of 
Applied Psychology, XXIV (1940), 595-604 
The author piesents a table of equivalent values foi 1 Q ’s 
fiom the 1916 and 1937 revisions of the Stanfoid-Binet and 
suggests a new classification of I Q ’s based on the 1937 foim 
The method by which the equivalency was calculated is dis- 
cussed The suggested classification of I Q "s provides a senes 
of equal steps oi gradations of brightness, IF A Varvel 

Dongan, K E and Goiy, A E “Selecting Unskilled Laborers 
m Cincinnati ” Public Pei somel Review, I 3 (1940), 
43-50 

Job analyses weie made of jobs foi unskilled laboiers 
as eligible lists became needed It was agieed that the ability 
to lead and write, a good physique, intelligence, expeiience, 
and an age lange of 21 to 45 or 50 weie lequiied foi the jobs 
An examination fot waste collectoi included a piactical 
test calling for lepeatmg a demoiistiation given by legular 
woikers, an evaluation of tiaining and expeiience, and an oral 
Intel view A test foi stieet cleaneis was composed of 75 
multiple-choice items on arithmetic, vocabulaiy, reasoning, and 
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general infoimation These questions weie put in the language 
of laborers 

Examining foi unskilled labor positions has gone on only 
since Febiuary, 1940 The departments, howevei, believe 
they aie getting bettei workers 

Dressel, Paul L “Some Remarks on the Kuder -Richardson 
Reliability Coefficient” Psychometnka, V (1940), 305- 
10 

The Kudei-Richardson reliability coefficient is derived in 
a mannei independent of that originally given. Vaiious altei- 
native forms applicable to special situations aie exhibited 
With the pul pose of making them available to others mteiested 
in using this formula A simplification in computation is sug- 
gested for use with a calculating machine (Courtesy 
Psychometnka ) 

Feiguson, Geoige A “The Application of Sheppaid's Cor- 
rection for Grouping” Psychoiiietnka, VI (1941), 21-7 
This papei attempts to show m a non-mathematical way 
the mfliience of grouping on standard deviations and con Na- 
tions, and advances empirical evidence to illiistiate with what 
accuracy values collected for gioiiping by Sheppaid's collec- 
tion approximate values obtained fiom ungiouped data when 
the distributions are continuous This inquuy gained its initial 
stimulus from the obseivation that many standard deviations 
and correlations reported by students of psychology and edu- 
cation are imcoirected for grouping and that fiequently errors 
attiibuted to the gioupmg of data are not small when com- 
pared with errors of sampling (Courtesy Psychowetuka ) 

Godard, R H, and Lindquist, E F “An Empirical Study of 
the Effect of Heterogeneous Within-Groups Variance upon 
Certain F-Tests of Significance in Analysis of Variance ” 
Psychometnka^ V (1940), 263-74 

In the application of the analysis of variance to data 
obtained in educational methods experiments which involve 
several classes of several schools, one assumption is that of 
homogeneity in the variances of pupil scores from school to 
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school It IS shown that such vaiiances on lepiesentative 
educational achievement tests aic hetciogeneons The effects 
of this hctciogejieity upon the F-tcsts of significance commonly 
employed in methods expeiiments are Investigated by com- 
paiing the actual distiibution of F values foi a laige number 
of "expeiiments” involving maiked heteiogeneity with a 
theoietical distiibiition based on the assumption of homo- 
geneity Although the findings, which vaiy somewhat with the 
type of vaiiance latio, aie not entiiely conclusive, they appai- 
cntly demonstiate that depaituie fiom homogeneity does not 
invalidate the use of the ciistomaiy F-tests foi evaluating 
results of the typical methods expeiiment (Courtesy 
Psychometuka ) 

Goodenough, Floience L and Mauiei, Katharine M "The 
Relative Potency of the Niiiseiy School and the Statis- 
tical Laboiatoiy in Boosting the IQ’* Journal of Educa- 
tional Psychology, XXXI (1940), 541-49 
This study recomputed data obtained at the Minnesota 
Nuisciy School by those statistical piocediues geneially 
eniployed in the Iowa statistical laboratory In the Iowa 
procedure, cases were grouped according to initial I Q. instead 
of paternal occupation This recomputation of data, which 
when handled properly showed no effect of nursery school 
training upon the I Q , gave results similar to those reported 
from Iowa A difference in I Q appeared foi children who 
remained at home as well as for nursery school children The 
authors conclude that the previously reported differences are 
the result of fallacious statistical treatment rather than being 
an educational phenomenon D A Peteison 

Guilfoid, J P "The Phi Coefficient and Chi Square as Indices 
of Item Validity,” Psychomeinka,W (1941), 11-9 
Two new methods of item analysis are desciibed One 
involves the computation of the 0 coefficient (coiielation of 
a fourfold point distribution) and the other involves chi 
square The only data required aie the proportions of passing 
individuals in the upper and lower criterion groups, for the 
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deteiminntion of 0, and m addition, N, foi the deteimination 
of chi squaie Abacs aie piesented foi giaphic solution of the 
two indices of validity, and tests of significance aie piovided 
(Conitesy Psychomeh ika ) 


Jenkins, R L “Consideiations Relative to the Selection of an 
Index of Intelligence” Joutnal of Edticaitonal Psychol- 
ogy, XXXI (1940), 527-40 

The test -1 etest stability of the I Q and the P C (Heinis 
personal constant) aie compaied in terms of Binet test latings 
for 1,774 cases The group was weighted with letaided 
childien Compaiisons of all adjacent tests weie made 
Regiession of both IQ’s and PC’s toward the mean on 
1 etest was found with maiked diops in the P C.’s of very 
bright childien “The PC, appeals to offer no advantage 
ovei the I Q foi the childien of the middle-age gioup ” 
and appeals to be slightly jnfeiioi to the I Q at the lower 
age levels 

The lationale uiideilying the two statistics aie considered 
Dispeisions in intelligence aie assumed in both cases to be 
piopoitional to the mental age The growth function assumed 
by the I Q and the PC aie presented with the point that 
both ciuves have one degiee of fieedom 

It IS suggested that a more logical appioach would be to 
express “mental test peifoimance in terms of the sigma value 
of test score foi the chronological age ” The assumption is 
less restrictive than those foi the constancy of the I Q oi 
P C ; the assumption is “that the relative status of children 
with respect to intelligence remains constant,” which is 
“implicit in the use of any index of intelligence foi piedictive 
purposes ” This index avoids the logical fallacy involved in 
adult mental ages It is pointed out that the giowth function 
may be a two-parameter cnive which would not inteifeie with 
the use of sigma values, but would i educe the value of a single 
paiameter statistic Harold Bechtoldt 
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Page, J D “The Eftect of Nuiseiy-School Attendance Upon 
Subsequent IQ” Joiunal of Psychology, X (1940), 
221-30 

Staiifoid-Binet I Q *s of 72 childien in kindcigaiten to the 
fifth giade who had pievioiisly attended niiiseiy school 125 
to 525 days weie compaicd with those of adjacent oldei sib- 
lings who had not attended pieschool. One bundled children 
of like age and socio-economic status wcie also compaied with 
then adjacent oldei siblings, none of eithei group having 
attended pieschool No significant diffeiences in I Q could 
be icferied to miiseiy-school attendance A slight advantage 
of youngei siblings in both expeiimcntal and contiol gioups 
was explained by age fluctuations in the standaidixation of 
the L foim of the Stanfoid-Binct No lelation was found 
between duiatioii of niuseiy-school attendance and subsequent 
IQ advantage The mean I.Q, difkeience between sibling 
pairs appioximated 10 points JV A Faivel 

Powell, N J “Check List foi Use m Civil Seivice Objective 
Test Piepaiation ” Public Pet sound Quaiteily, II (1940- 
41), 13-6 

The article includes a list of questions which have been 
developed foi leviewing civil seivicc objective tests before 
they ale finally used Its use is intended to “inciease the piob- 
ability that no majoi basis upon which the test will be 
appiaised has been ignored in the test construction “ Points 
to be checked are listed under the following headings validity, 
cost, appearance of test, typography, and administration A 
numbei of questions applying specifically to completion items 
and multiple-choice items are also listed 

Roff) Merrill “Lineai Dependence m Multiple Coiielation 
Work “ Psychometnka, V (1940), 295-98 
The problem in multiple correlation work of nonsense 
lesults attributable to linear dependence of vaiiables, which 
has been discussed by Ragnar Frisch in relation to economic 
data, IS presented from the standpoint of its significance in 
psychological reseaich It is shown that a symmetric coiie- 
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lation deteiminant with unity in the diagonal cells can vanish 
only when theie is a fiist-oidei oi paitial con elation of unity 
between one pan of the vaiiables On the basis of this lesuJt, 
It IS aigued that the pioblem should be expected to cause less 
difficulty in the field of psychology than in economics and that 
psychologists should be able to avoid the pitfall by bunging 
to beai then knowledge of the vaiiables with which they are 
working (Couitesy PsychoineUtka ) 


Royei, Elmei B “A Machine Method for Computing the 
Biseiial Coi relation Coefficient m Item Validation’* 
Psychomeuika, VI (1941), 55-9 

A method foi computing the biseiial con elation coefficient 
with the aid of punch-caid equipment Is outlined A numer- 
ical example and a woik sheet layout are included in the 
piesentation (Couitesy Psychoinetrika.) 

Ryans, David G The Fust Step in Guidance Self-Appraisal 
New York, Coopeiative Test Seivice 35 pp 1941 
Aiepoit of the 1940 Sophomoie testing piogram in which 
the following tests were used Coopeiative English Test, Form 
Q, Coopeiative Geneial Cultuie Test, Foim Q, and Coopeia- 
tive Contempoiary Affans Test, Foim 1940 


Sisk, H L “A Note on the Compaiative Value of the 'True* 
Index of Studiousness foi the Piupose of Prognosis ” 
Journal of Psychology^ X (1940), 275-78 
The scholastic achievement of 585 univeisity fieshmen was 
predicted from Symond’s “true” Index of Studiousness and 
from a battery of tests, composed of aptitude, English, and 
leading The latter was found to give a moie leliable pie- 
diction of first semestei grades PF A Fmvel 


Stoy, E G “Selection of Key-Punch Opeiatois ” Journal of 
Applied Psychology^ XXIV (1940), 653-54 
These aie notes on prehminaiy experimentation in the 
selection of key-punch opeiators Foiii tests wairant fuither 

213 



lducationai and psychological measurement 

coiisideintion “an eye-hand cooidiiiation test in which lettei 
combinations mvoLvmg both hands aie legistered on counters, 
a test of veibal and spatial niemoiy, a cleiical type of test, and 
an authmetic test ” JF d Vawel 

Swinefoid, Fiances and Hobnngei, Kail J “Selected Refer- 
ences on Statistics, the Theoiy of Test Constiuction, and 
Factor Analysis” School Review, XLVIll (1940), 
460-66 

Ai ticks coveiing the year Maich, 1939, to February, 
1940, aie picsented with biicf notes as to the nature of the 
pioblem handled in each papei Twelve ai tides aie given 
under the heading “Theoiy and Use of Statistical Mehods,” 
18 undei “Pioblems of Test Constuiction,” and 16 under 
“Factoi Analysis,” Haioid Bechtold^ 

Thill stone, L L “A Factoiial Study o£ Visual Gestalt 
Effects” Psychomeii ika, V (1940), 31546 (Abstract 
of a papei icad at the September, 1940, meeting of the 
Ameiican Psychological Association ) 

Toolon, W T “Essential Factois in Test Construction” 
Petsonnel Jotntial, XVIV (1940), 204-08 
The value of caieful “informal examination” of test items 
befoie and aftei statistical tieatment is pointed out, and an 
analysis of the natuie of the items and of the eiiors made 
IS suggested Factois dealt with include item difficulty, item 
couelations, closeness of disti actors, and the judgment and 
information of the subject Hat old Bechtoldt 

Tucker, Ledyard R “A Matux Multipliei ” Psychometiika, 
V (1940), 289-94 

A machine to expedite matiix multiplication has been 
developed by modifying the International Business Machines 
Corporation scoiing machine The principles and operation 
of the machine aie described, and time and accuracy estimates 
aie indicated (Coiutesy Psychometnka ) 
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A NEW PERFORMANCE TEST FOR YOUNG 

DEAF CHILDREN 

MARSHALLS HISKEY' 

University of Nebraska 

Inti oduciion 

T here has long been a need for a measuring device 
which would give the teachei of the very young deaf 
child a valid indication of his learning level at the begin- 
ning of his educational career One can find a consider- 
able numbei of moie or less carefully worked out mental 
tests which have been used for deaf and hard-of-hearing 
individuals However, the degiee of help which such tests 
lender the educatoi or clinician depends upon the num- 
ber and lepresentativeness of the children upon whom 
they have been standardized, and also upon the reliability 
and amount of mfoimation about the children which the 
test makes available Few tests have been standardized 
on deaf children or used with such children at the begin- 
ning of their school experience 

Instiuction, especially at the lower levels, although 
carried on as a group activity, actually involves consider- 
able individualized work This individualization is, m 
many instances, primarily a means of prepaiing foi group 
instruction Theiefoie, if classes are not composed of 
students of approximately the same level of ability, the 

^The writer wishes lo acknowledge his indebtqdntsa to Dr DA Worcester 
and other ataS members of the Department of Educational Psychology and Meas- 
uiemcnts of the University of Nebrnsksi and to the administrations of the Iowa, 
Nebraska, Kansas, Missouu, Illinois, Indiana, and Ohio Slate Schools for the 
Deaf 
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teachei must spend entirely too much of her time working 
with the slower pupils as individuals In many instances 
this IS done at the expense of the more capable students 
and often results in a gicat waste of time since it is diffi- 
cult to keep the young deal child occupied constiuctively 
without the diicct, and almost constant, guidance and 
supei vision of the teacliei, If supplementary measuring 
devices aie valuable in making the school program more 
effective for the heaimg child, then they should be even 
moie valuable with a gioup who must start with the 
handicap of deafness 

Difficulties involved in conshnehng a test for the 
young deaf and haid-of-heai mg. In the selection of items 
for young deaf children, the special limitations of this 
group must be kept constantly in mind The actual test- 
ing of deaf childien piesents problems which are unique 
Piactically eveiy impression of the test materials gained 
by the deaf child must be thiough the sense of sight All 
instiuctions must be given through pantomime Because 
of the child^s complete lack of language expeiience, the 
test items must have an unusual intrinsic attractiveness 
In addition to these problems one must devise a sufficient 
variety of items to sample adequately the abilities of indi- 
viduals whose range of experiences has been seriously 
restricted 

To attempt to obtain a rating of the ‘ Vord fluency” of 
the child who has been deaf since birth would be futile 
Nor does it seem appropriate to include speed tests since 
it IS very difficult to give to the young deaf child the con- 
cept of speed 

Based on the observations gained through testing the 
members of both groups, the writer is of the opinion that 
deaf subjects aie more prone to “jump to conclusions” 
and to ovetestUTiate their abilities or the amount of mate- 
rial which they have grasped, than are hearing subjects 
It IS necessary to make them take their allotted time for 
viewing materials before they attempt a response On the 
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Other hand, the examiner must always be an the alert, lest 
through some slight change in facial expiession he assist 
the subject in making his response The deaf or hard-of- 
hearing child is continuously seeking visual clues and an 
“arched eyebrow” or the “flickei of an eyelash” may 
speak volumes to him 

The wiitei has made no attempt to compare the intel- 
lectual development of deaf and hard-of-hearmg children 
with that of hearing children The deaf child’s training 
probably will never be identical with that of the hearing 
child. The writer is of the opinion that the question of 
primary importance is not, "How does the deaf child rank 
in comparison with the hearing child P”, but rather, 
"How does the deaf child rank in comparison with other 
deaf children of his chronological age?” 

Development and Standm dizahon of the Scale 

ehminmy study of deaf and hat d-of -heat i 7 iff chil- 
dten in school In oidei to obtain a moie adequate under- 
standing of the gioup, the writer made an intensive study 
of deaf children as they actually went about then school 
work 

For a peiiod of more than four months the writer 
spent three days every two weeks with these pupils in a 
residence school for the deaf Not only did he visit them 
at their class work but he lived with them at the school 
and associated with them on the playground, in the gym- 
nasium, and elsewhere, A complete record was made of 
the activities which took place in the classroom and also 
of those of an extia-curiicular nature This type of study 
yielded a multitude of suggestions which weie of the 
utmost importance in the constiuction of the scale 

The selection and constiuction of test items Every 
Item of the scale was considered in light of the following 
criteria: (1) Was the item similar to the task, oi tasks, 
which the young deaf child did m school? (2) Was it the 
type of item which could be included in a non-verbal 
test? (3) Could the item be presented in such away that 
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diiections could be given thiough simple pantomime? 
(4) Was It tbe type of item which experience had shown 
to yield high conelation with acceptable cnteiia of intel- 
ligence or learning ability? (5) Could the item be con- 
stiucted and presented in such a way that the child could 
give a definite response, thus making the scoiing objective 
and easily done? (6) Would the item be appealing oi 
atti active to the subject? (7) Could the item be scoied 
without the scoie being based on time? (8) Did the diffi- 
culty of the item appear to be within the age laiige of the 
standaidizmg group? (9) Did the item seem likely to 
show a high discriminative capacity? 

In many instances, in order to meet all the above 
criteria, it was necessaiy to devise special methods of 
constructing or assembling the paits of an item The pre- 
liminary scale was composed of 18 diffeient types of items 
with a total of 204 items 

The use of the pteliuiinaiy scale This scale was given 
to seventy-thiee pupils of the Towa School for the Deaf, 
whose ages langed fiom three years ten months to nine 
years eight months Owing to the length of the scale, it 
was divided into two paits and half of the group was 
given Part A first and the othei half of the group was 
given Part B fiist The two parts were given not less 
than one day noi more than one week part In several 
instances items weie scored m detail, thus permitting a 
later resconng on a different basis 

Aftei members of this tryout group were tested, an 
item analysis was made and curves of the percentage 
passing each successive chronological age were plotted. 
This was done for each of the 204 individual items of the 
scale The steepness of these cuives afforded a graphic 
indication of the validity of the items. The items which 
appeared to function the most satisfactorily and to most 
neatly approximate the criteria were retained. The cri- 
teua used weie (1) validity (based on the percentage 
passing from one age to the next) , (2) ease of adminis- 
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tenng, (3) ease and objectivity of scoring, (4) attractive- 
ness or inteiest to the subject, (5) variety; and (6) time 
of administering When the sifting process was com- 
pleted, 1 1 types of tests weie retained, including a total 
of 124 individual items 

The test items A buef description of the items may 
make latei discussions more meaningful The types aie 
as follows 

1 Mcmorj' for Colored Objects — ^Two sets of eight coloicd sticks 
each, one set foi the examinei and one set for the subject The 
examiner presents fiom one to five of the sticks from his group and 
then removes them and the subject must select the corresponding 
sticks from lus set fiom memory, 

2 Bead Stiinging — ^At the lowei levels scoiing is based on the number 
of beads stiung duiing a tw-o-minute period The intermediate 
level demands the coiiect copying of bead pattcins, while at the 
upper level the subject ts lated on Ins ability to repioduce patterns 
from memoiy 

3 Pictorial Associations — ^T!’his includes 12 senes of pictures , In 
each senes two pictiiies are mounted side by side and a recess is 
left for the insertion of the thud picture which is associated with the 
first two This third picture must be selected from a group of 
four unmounted pictures (There are four unmounted pictures foi 
each senes ) 

4 Block Patterns — A set of eight drawings of block patterns and 16 
blocks The patterns arc arranged in ordci of difficulty and the 
subject must construct the pattern shown in the di awing, 

5 Memory for Digits — ^Two <iets of nine numbers each The exam- 
iner presents a number scries and the subject must repioduce it from 
memory 

6 Completion of Drawings — ^A senes of 15 pictures, each with a part 
missing, The subject must draw the missing part and thus com- 
plete the picture 

7 Pictorial Identification — Six series of mounted pictures Each series 
has five picturc'i of a similar nature which are mounted side by 
side Four individual pictures which are duplicates of the mounted 
pictures must be correctly identified by matching them with the 
corresponding mounted picture 

8 Paper Folding — Six-inch squares of paper which must be folded 
by the subject to duplicate (seven) patterns 
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9 Visual Attention Span — Several scncti of pictures (varying froTti 
one to SIX pictuics each) and 15 individual pictures The subject 
IS shown a pictuie series and lie must use the individual pictures to 
lepioduce the presented scries from metnoiy 

10 PuKzlc Blocks — ^Eight sets of variously shaped pieces of wood Each 
set can be put together to form a block 

11 Pictoiial Analogies — Ten senes of plctuics with three pictures m 
each senes mounted and foiii pictures to use as choices The first, 
second, and third pictuics of the analogy aie mounted and a recess 
is left for the insertion of the foiiith picture which completes the 
analogy The subject must select the latter picture fioni among the 
foui available choices, 

Use of the pi ovisional scale In addition to the work 
d,one with the pupils of the Iowa school, the test was 
administered in the state schools foi the deaf in Nebraska, 
Kansas, Missouri, Illinois, Indiana, and Ohio, as well as 
to the members of the Lincoln, Nebraska, Day School 
All students, except a few who were ill, who were under 
10 or who had had their tenth biithday within IS days 
of the examination date were tested The test was admin- 
istered to 466 individuals. The standardizing group is 
limited in numbeis at the age of four and below, since 
most schools do not accept childi en until they are five or 
SIX years of age 

DenvaHon of the final scale. To save time and to 
guarantee greater accuracy in the statistical data on which 
the final selection of items would be based, Hollerith 
techniques were used By means of the Hollerith sorter 
and counter it was possible to determine quickly the num- 
ber of individuals who were successful on each item in 
successive ages throughout the lange and thus to plot for 
each Item the curves of percentage passing Items were 
selected chiefly on the basis of discriminative ability, this 
judgment being based on the increase in percentage pass- 
ing from one age to the next The items in each group 
were next arranged in order of difRculty, this order being 
based on the percentage of the total group passing each 
individual item 


222 



PERFORMAKCE test for deaf CFI[LDREN 


To develop the table of norms, curves were plotted 
showing for each age gioup the percentages making each 
possible total score foi each group of items The score 
necessary for passing each item at a certain age level was 
considered to be that score which was made by approxi- 
mately 70 per cent of the particular gioup In all in- 
stances the peicentages were plotted, the curves were 
smoothed, and the ends were extended to obtain what 
might he termed “piojected norms” at the extremes This 
smoothing of the curves of percentages gives a somewhat 
truer indication of the ability level of the four-year-old 
group 

To determine who should compose the four-year-old 
group, 01 the five-year-old group, etc,, it was decided to 
classify all individuals as four whose ages were behveen 
three years six months and four years five months and as 
five those who were between four years six months and 
five years five months, and so on, In no instance does the 
mean chronological age of the standardizing group devi- 
ate more than one month from the desired or true mean. 

The unit of measurement Peihaps the most common 
method of interpreting scores on a scale such as this one 
IS the familiar Bmet type mental age This is the method 
of using age norms and the amount of mental develop- 
ment in a year as the unit of measurement. Age norms 
are established for raw scores and are converted into, or 
interpreted as, mental ages. This age- type score, repre- 
senting the amount of development up to date, has much 
greater meaning to the layman than does the ^^standard 
score” or the ^'percentile score” and for that reason the 
age norm has been used in this scale However, the term 
“mental age” has not been used because the M A would 
undoubtedly suggest a Binet Mental Age which in tuin 
would suggest the corresponding M A of the hearing 
child and thus lead to false comparisons For this reason 
and because of the fact that the test items have been 
selected, in many instances, because of their similarity to 
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the abilities which the deaf child must exhibit m school, 
the term ''Learning Age” is used instead 

An L A. of 5-0 simply means that, according to the 
results of this test, the child is able to do those tasks which 
the average deaf child of five yeais is able to do, or, that 
he should be able to solve problems with the same average 
efficiency as the average deaf five-year old 

It IS recommended that in the inteipietation of test 
results, the learning age be used instead of the learning 
quotient (L»Q , derived by dividing the LA by the C A , 
similai to the I Q ) Until moie conclusive evidence 
regarding the respective influence of environment and 
heredity on the mental development of the child, resulting 
from more carefully conti oiled experiments, is produced, 
one must proceed cautiously to msuie that he is not clos- 
ing the door of oppoitunity to any child If there is a 
reasonable question as to whether the hearing child can 
be improved through a stimulating progiam of training, 
IS it not likely that this question will assume even larger 
propoitions m the case of the deaf child? 

Stahsiical Analysis of Test Data 
The accuiacy of any test is dependent, not only upon 
test items employed, but also upon the number of indi- 
viduals examined, the representativeness of the group, the 
accuracy with which the test has been scored, the deriva- 
tion of accurate and meaningful norms, and various sta- 
tistical applications which aie used for the purpose of 
checking, or for interpretation An additional and more 
detailed statistical treatment of the data will be made at 
some latei date Such topics as sex differences, effects of 
schooling, relation of scoie to degree of hearing loss, 
resemblances of scoie to teachei judgment of ability, and 
the results of a factorial analysis of the test items, aie 
among those so reserved 

Adequacy of the siandaidizattoih Perhaps the mam 
criterion for the standardizing of any test is the selection 
of representative populations at each age The method 
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employed in meeting this problem has been described 
briefly above, i e , the testing of all available pupils 
(within the desired age range) in a rather widely scat- 
tered group of state schools foi the deaf. However, to 
check the adequacy of the sampling of cases, a table of 
peicentages of scores for each item v^as made which did 
not include the students of the Indiana school and the 
Ohio school Fiom this list of percentages, a table of 
norms was deiived These noims were then compared 
with the norms derived from the total group In 89 
per cent of the cases, the noims weie found to be iden- 
tically located and in the remaining 11 per cent of the 
cases they varied not more than six months This would 
indicate that the sampling was probably sufficient for 
determining relatively stable norms 

TABLE 1 

YEARS-IN- SCHOOL DISTRIBUTION BY AGES AND 
A COMPARISON or IHE MEAN CA'a AND THE MEAN LA's 
FOR THE STANDARDIZING GROUP 


Yeirs Jn School 

Age 

0 

1 

2 

3 

4 _ 

5 

6 

Total Mean C A 

MeanL A 

+ 

9 

1 






10 

4-1 

448 

5 

33 

9 






42 

5-0 7 

5-1 8 

6 

39 

16 

4 

1 




60 

6-0 3 

fi-3 5 

7 

Z2 

42 

15 

4 

1 



84 

7-0 

7-2 3 

a 

11 

31 

27 

14 

4 



S? 

7-11 + 

8-0 7 

9 

6 

29 

43 

31 

5 

3 


117 

8-11 7 

9-2 9 

9-9 

5 

9 

15 

17 

12 

6 

2 

66 

9-9 

9-6 5 

Total 

125 

13? 

104 

67 

22 ~ 

9 

2 

456 




Table 1 gives the number of individuals tested at each 
chronological age level The small number of cases in the 
lowest age group means that the noims will be less reli- 
able at the age of four This table also shows the mean 
chronological age and the mean learning age for each of 
the age groups In no instance does the mean chronolog- 
ical age differ moie than one month from the desired 
chronological age The mean leainmg ages likewise cor- 
respond closely to the mean chronological age At each 
age level, except one, the mean L A is slightly higher 
than the mean C.A It is felt that this is a desirable fea- 
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table 2 

PER CENT OF EACH AGE GROUP 


BEAD STRIHCIUG II OCK nUELDING 


Total per 2 Min 

Age 7-8 ?-l0 11-12 13-U 

Total Palteriis 
I II III IV 

V 


2 

Total Score 

3 4 5 6 

7 

8 

+ 

100 

90 

40 30 

40 

10 




100 

lOO 

30 

10 

10 




S 

100 

91 

79 55 

60 

29 

12 



100 

100 

SI 

40 

17 

2 



6 

100 

97 

9+ 87 

92 

47 

20 

5 


100 

100 

98 

82 

52 

27 

8 

3 

7 

100 

99 

99 97 

96 

87 

45 

12 

2 

100 

99 

99 

90 

73 

48 

24 

6 

8 

100 

100 

100 100 

100 

95 

67 

32 

9 

100 

100 

99 

94 

85 

69 

44 

li 

9 

100 

100 

100 100 

100 

99 

88 

57 

19 

100 

100 

99 

98 

95 

83 

66 

39 

9-9 

100 

100 

100 100 

100 

100 

89 

_70_ 

21- 

100 

100 

100 

100 

98 

95 

80 

50 




PICTORIAL 

ASaOCTATlONE 






PAPER FOLDINC 





Total Score 







Total Score 



Age 

1 

2 

3 4 5 

6 

7 

S 

9 

10 

11 

12 1 

2 

3 

4 

S 

6 


+ 

100 

lOO 

100 90 60 

20 

20 





100 100 

100 

40 

10 



5 

100 

100 

100 93 79 

40 

26 

12 

2 



100 100 

100 

88 

71 

26 


6 

100 100 100 97 90 

80 

62 

43 

20 

8 

2 

100 100 

100 

97 

90 

72 

16 

7 

100 100 

99 99 95 

89 

83 

67 

54 

23 

6 

1 100 100 IDO 

98 

95 

89 

61 

3 

100 

100 

100 99 99 

97 

94 

85 

67 

SI 

18 

6 100 100 

100 

99 

99 

96 

69 

9 

100 100 

100 lOO 99 

99 

97 

96 

82 

73 

39 

12 100 100 100 

100 100 

99 

91 

1 

100 

100 

100 100 100 100 100 : 

100 

91 

82 

58 

14 100 100 

1 100 100 IQO 100 

95 




MEMORY 

rOH. COIQUO QDJECnS 




MEMORY FOE 

Age 5 6 

7-8 

9 

Total Score 
10 11 12 

13 

14 

15 

16 

17 

Part A Totals Not in Order 
3 4 5 D C D E 

4 

100 

90 

40 

20 

10 







100 80 10 10 




5 

100 

98 

62 

26 

14 

2 

2 





IDO 88 45 64 

ID 

ID 

ID 

6 

100 

98 

90 

70 

52 

27 

15 

S 

2 



100 95 87 97 

65 

ID 

10 

7 

100 

99 

98 

89 

77 

54 

31 

13 

6 

1 


100 99 97 98 

33 

38 

15 

8 

100 

100 

99 

93 

84 

71 

60 

40 

22 

11 

3 

100 100 100 100 

94 

59 

17 

9 

100 

100 

100 

97 

91 

85 

73 

56 

43 

27 

5 

100 100 100 99 

94 

79 

37 

9'9 

100 

100 

100 

100 

100 

95 

BO 

62 

48 

29 

17 

100 100 100 100 

98 

83 

52 


ture If there were an inadequate sampling of subjects, 
it would likely be of the group with limited ability, since 
the mentally less advanced aie less likely to have entered 
school at a reasonably eaily age than are those mentally 
advanced The test ceiling is not high and this may be 
responsible for the fact that the gioup at the upper end 
of the range has a mean L A slightly below their mean 
C A In only two instances do the two means deviate by 
as much as three months — the greatest deviation being 3 8 
months at the four -year level. 
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MAKING EACH SCORE ON EACH TYPE OF ITEM 


pictorial IDENTinCAlION VISUAL ATTEtfTIOH SPAN 


'lotal Sc-ore 

1 2 3-+ 5-6 7-B 9-10 11-12 13^14 15-16 17-13 19 20 21-22 23-24 1 

Toip] 
2 3 

Score 

4 5 

6 

IQO 100 100 100 

100 

90 

80 

50 

40 

10 



100 

70 

20 

10 



iOfl 100 100 IDO 

93 

95 

93 

79 

60 

33 

10 

5 

lOO 

73 

29 

2 



100 IQO 100 100 

100 

98 

93 

80 

65 

48 

29 

23 

100 

95 

72 

35 

12 

2 

loo 100 100 100 

99 

99 

98 

96 

94 

85 

75 

60 

100 

99 

90 

57 

21 

3 

!00 100 100 100 

100 

100 

100 

99 

98 

98 

95 

87 

lOO 

100 

9B 

77 

36 

11 

IQO IQO 100 100 

100 

99 

99 

98 

98 

98 

98 

93 

IQO 

100 

98 

85 

51 

26 

100 100 100 100 

100 

100 

100 

100 

100 

100 

100 

98 

100 

100 

100 

88 

70 

33 




PUZZLE nLGCK3 






PICTORIAL 

ANALOGIES 



1 

2 

Poial Score 

3 4 5 

6 

7 

1 

2 

3 

Total 

4 5 

Score 

6 

7 

8 

9 

10 

90 

30 






10 

10 

10 

10 







100 

95 

55 

13 




100 

76 

71 

43 

14 

5 

2 




too 

98 

80 

52 

12 



100 

100 

95 

60 

57 

30 

12 

2 

2 

2 

100 

98 

87 

73 

38 

5 


100 

99 

96 

89 

79 

52 

27 

10 

1 


100 

100 

99 

89 

64 

22 

2 

100 

100 

100 

98 

87 

63 

43 

28 

2 


100 

100 

100 

97 

77 

54 

14 

100 

100 

100 

98 

94 

91 

76 

56 

26 

7 

100 

100 

100 

100 

89 

63 

21 

100 

100 

100 

100 

98 

95 

83 

64 

30 

S 


DIGITS 






COMPLETION OF 

DRAWINGS 







B 

In Order 

C D E 

__ 1 _ 

2 

_3 

4 

5 

Total Score 

6 7 8 

9 

10 

11 

12 

13 

14 





20 

10 

10 












50 

2 



67 

44 

36 

19 

17 

12 

7 

2 

2 

2 





88 

33 

3 


98 

93 

87 

77 

70 

55 

38 

25 

20 

10 

7 

2 



91 

55 

13 


98 

96 

94 

87 

85 

77 

73 

67 

52 

43 

30 

19 

7 


98 

SO 

32 

6 

100 

100 

99 

98 

95 

95 

94 

92 

69 

74 

53 

34 

16 

7 

98 

83 

S3 

24 

100 

100 

100 

100 

99 

99 

97 

96 

88 

86 

79 

71 

47 

16 

too 

97 

73 

32 

100 

100 

100 

100 

100 

100 

100 

100 

100 

97 

94 

83 

61 

23 


Table 2 gives the percentages of subjects in each age 
group making each possible total score for each type of 
test item These per cents were the onefe used m plotting 
the cuives of per cents These curves were smoothed and 
the scote which revealed approximately 70 per cent of 
success was taken as the score of the average person of 
that chronological age It was then entered m the table 
of norms under that learning age The learning ages 
given below 4-0 and above 10-0 are the results of an exten- 
sion process and, as has been mentioned before, are not so 
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reliable as aie those within the age range of the subjects 
examined 

Validity The veiy methods by which the test items 
have been selected and letained aie evidence of their 
validity It will be lecalled that the items were selected 
according to rathei definite criteria and that after they 
had been given they weie subjected to a iigoious item 
analysis Thus the chief ciiteiia for validity were (1) 
selection — thiough cutical analysis and adherence to cri- 
teria, and (2) inciease in the peicentage passing from 
one age to the next In the piesent scale, it was impossible 
to deteimme validity through coi relations with other test 
scores inasmuch as there is no existing test which would 
have been an acceptable ciiterion In the absence of the 
needed ciiteiion, coriclations weie computed between 
the score on the entire scale (the scoie on the entire scale 
IS the median learning age of the learning ages obtained 
on the several paits of the scale) and the score on each 
group of items The coi relation setup is seemingly a 
spurious one since a part of the test has been corre- 
lated with the whole test which includes this part As 
the score on the entiie scale is the median score of the 
parts of the scale, however, each part has an approxi- 
mately equal share in producing this total or final score 
and this in turn lessens or eliminates the possibility of 

TADIE 3 

CORRELATIONS BETWEEN THE LEARNING AGE OBTAINED ON ONE 
SECTION OF THE TEST AND THE MEDIAN LEARNING AGE 
OBTAINED ON THE ENTIRE TEST 




Group I 
(Age 4 to 7) 

Gioup 11 
(Age 8 to 10) 

1 

Memory for Colored Objects 

S10+ 

740 

2 

Bead Stiinging 

812 

729 

3 

Pyc-tornl Asaocutions 


m 

4 

Block Building' 

797 

718 

5 

Memory fox Digits 

755 

773 

6 

Completion of Drawinga 


702 

7 

Pictorial Identification 

730 


i 

Paper Folding 

843 


9 

Visual Attention Span 

637 

629 

10 

Puzzle Blocks 

i 

734 

U 

Pictorial Analogies 

f 

742 
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the obtained corielations being spuriously high Since 
the correlations between the learning age obtained on each 
group of Items and the median learning age on the entire 
scale aie within the range of fiom 629 to 843, they are 
evidence of high internal consistency and thus, perhaps, 
of high item validity 

The abbteviated scale, To determine whether a de- 
pendable short scale could be assembled, the five types of 
items which showed the highest con elation with the me- 
dian leaining age foi the entire scale were selected to form 
the abbreviated scale Since some of the groups of items 
do not function over the entire age range, correlations 
were derived separately foi two groups Group f was 
composed of all membeis of the standardizing gioup who 
weie seven yeais or under, and Group II, those who were 
from eight to 10 years of age For Group I, correlations 
With the total scale weie obtained for all gioups of items 
except those which do not function at the lower levels, and 
for Gioup II, correlations with the total scale were ob- 
tained for all groups of items except those which do not 
function at the higher levels The best booklets were 
rescored on the basis of these abbreviated scales and cor- 
relations were found between the median learning ages 
obtained from the abbreviated scales and the original 
scale The correlation for Group I was 944 and for 
Group II 936 Thus, when time limitations make it nec- 
essary, the short forms may be used with a considerable 
degree of confidence These abbieviated forms can be 
given in approximately 30 minutes 

Although it IS recommended that the learning age be 
used in preference to the learning quotient (L Q = 
L A / C,A ), in order to make the study more complete 
and significant the writer has made a study of the L Q 's 
of the standardizing group Table 4 shows that the mean 
learning quotients derived from the standardizing group 
closely approximate the desired mean of 100 The great- 
est deviation is at age four and is probably due to the 
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small number of cases and to the fact that they are a 
somewhat select group 


tadi b + 

THE MEAN LEARNING QUOTIENT* RANGE* STANDARD DEVIATION 
OF THE LQ's, AND STANDARD ERROR OF THE MEAN FOR EACH 
AGE LEVEL OF THE STANDARDIZATION GROUP 


Age 

Menn 

LA 

RTnge of 
LO's 

c 

LQ 

a 

M 

+ 

108 $ 

94-127 

10 909 

3 +500 

5 

102 7 

80-12+ 

10 518 

1 6228 

6 

10+4 

65-139 

14+70 

1 8681 

7 

m 4 

+3-132 

15 300 

15912 

8 

101 7 

65-137 

15 135 

1 6226 

9 

10+0 

55-13+ 

14361 

1 3277 

9-9 

99 1 

73-120 

11410 

14040 


The mean L.Q at each age level except the upper 
group (9-6 to 10-0) is slightly above 100. As has been 
mentioned before, this is piobably a desirable feature 
The lower mean of the upper gioup is apparently the re- 
sult of the limited test ceiling The standaid deviations at 
each age agree closely, except at the two extremes where 
the attenuating factors befoie mentioned have influenced 
them In general, the standard deviation of the means is 
approximately 1 6 (disregarding the four-year group) 
Every effort has been made to make the test usable 
and yet have the mechanics as simple as possible The 
record blank (Table 5) has been no exception and has 
been patterned after the one devised by Hildreth and 
Pintner. The record blank is m reality a table of norms 
and the various scores are checked on the blank and the 
median score is calculated* The items of the abbreviated 
scales also are indicated on the blank 

The test items not only aie attractive to young deaf 
children but also they have a rather high discriminative 
value The scale is not difficult to administei or score and 
since It is weighted heavily with tasks similar to those 
which the deaf child must do in the early years of his edu- 
cational career, it should be extremely valuable for gam- 
ing a better understanding of the abilities of the younger 
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deaf cliildreii. This is not intended to imply that the 
inexperienced person could give the test satisfactorily, 
The person who is unfamiliar with individual testing 
techniques would have considerable difficulty unless he 
underwent a peiiod of tiaining or practice with this scale 
It IS quite conceivable that the person who has had some 
experience in individual testing and who has some knowl- 
edge of deaf children could, after a period of training in 
which he gave six oi eight practice tests, administer the 
scale quite satisfactorily 
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PERFORMANCE TESTING IN PUBLIC 
PERSONNEL SELECTION 

PART I 

SIDNEY W K0RAN‘ 

liiTiployment Boards Fennsy1v^^la Department of Public Assistance 


InU eduction 

I T IS an inteiesting fact that although the use of per- 
formance tests m the selection of public personnel 
enjoys not only the geneial endoisementof personnel tech- 
nicians but the enthusiastic and unsolicited support of the 
public as well, theie is probably no other aspect of the 
examination process at present more completely neglected 
by the majority of merit system agencies 

Probably all jurisdictions employ performance tests 
in the selection of typists and stenographers, and the gem 
eral practice is to convert the ratings on these tests into 
quantitative terms capable of combination with scores 
achieved in othei portions of the examination battery 
Beyond that, however, the performance testing of most 
agencies seems seldom to go beyond the administration of 
qualifying tests to a sufficient number of individuals at 
the top of certain registers to satisfy immediate certifica- 
tion requirements Except for the case of tests of typing 

iThe flutlior deairta ta express hia appreciation to the fullowing individuals 
Mrs Ruth Glenn Pennell^ and Mi Robert Hall Criig, members of the Employ- 
ment Boirti, Miss Hilda P Thompson, the Boaid's Executive Director, Dr C 
II Smeltzcr, the Board's Technical Consultant, Miss Kathleen Oyster, FrafRc 
Representative of the Bell Telephone Company’s Hairisburg office, Mr Andrew 
S Hay, Service Supervisor of the IBM Harrisburg office, Mr Bernard Gehnng 
of the Miiltigraph Sales Ageini|r m Harrisburg, and Miss Alice I Thompson, 
of the Penn State Alumni Association 
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and stenography, the technique of using the performance 
test as a major pat f of the test battery — that is, as a factor 
which may decidedly influence an examinee’s relative 
standing on the eligibility register — appears to have been 
almost completely ignored 

There ate, of course, vaiious reasons why this situation 
exists Associated with the technical difHculties inherent 
in the construction and administration of the performance 
test — difficulties which, incidentally, aie fiequently not 
nearly so “insurmountable'’ as they at first appear — may 
be the factois of cost and already overburdened technical 
staffs In addition, newly created agencies frequently face 
time deadlines which all but preclude their going beyond 
the commonly accepted minimum selection elements, 
namely the use of minimum requirements, a written test, 
an evaluation of training and experience, and, for certain 
positions, an oial interview In addition to these factors, 
howevei, and probably overshadowing them in effect, 
must be mentioned two others general inertia and, prob- 
ably very closely related, an uncritical adherence to time- 
honored examination patterns considered satisfactory in 
selecting persons for jobs not requiiing the possession of 
manual skills 

Considerable progress has already been made in the de- 
velopment of peifonnance tests for the selection of typists 
and stenographers Since their use is so widespread, no 
further attention will be devoted to them in this discus- 
sion beyond pointing out that, despite their popularity, 
much necessary work still remains to be done toward their 
improvement, especially in the development of (1) suit- 
able standards of performance, (2) satisfactory scoring 
procedures, and (3) improved standardized techniques 
for administeiing the stenographic portion of the test,* 

^The Chicago Park District's use of [ihonographio recordings and the novel 
experiments of the Bul^alo Municipal Civil Service Commission and the Arizona 
UnemplDymcnt Compensation Merit SysLem Council vrith radio broadcasting are 
examples of approaches to the problem of minimizing or eliminating the un- 
desirable effects of varying dictation speeds and other factors which characterize 
the use of numerous proctors 
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Techniques have also been developed for measuring 
peitormance m othei jobs such as chauffeur and m certain 
skilled trades Many companies test applicants for chauf- 
feur positions and most state motor vehicle bureaus give 
qualifying driving tests to applicants for operator licenses 
The latter aie ordinarily quite informally conducted, but 
Viteles has descnbed “a trade test of driving skill’*® which 
could quite leadily be adapted to meiit system use.^ 

The New York City Civil Service Commission’s ex- 
cellent pioneer work in developing tests for such skilled 
trades positions as welder, machinist, electrician, lock- 
smith, lineman, and carpentei is quite well known ° 
Recently the State Technical Advisory Service of the 
Social Secunty Board began work on standardizing a 
perfoimance test for Key Punch Operators 

In general, however, the use of the peiformance test 
as a measuring instrument designed to serve not only as 
a qualifying hurdle, but also as an impoitant factor in 
determining the examinee’s lelative standing on the eli- 
gibility register has received much less attention than it 
deserves The dearth of literature on the subject attests to 
this and probably contributes to the widely held feeling 
that perfoimance tests capable of producing quantitative 
ratings are somehow exceptionally difficult to prepare 
and impractical to administer 

It is hoped that this presentation will illustrate some 
of the possibilities by describing four actual tests used 
successfully by a medium-sized merit system agency, the 
Employment Board of the Pennsylvania Department of 
Public Assistance Small jurisdictions may be able to use 
some of the material with few or no changes Larger 
agencies, especially those with adequate technical staffs, 

S Viteles, biduslnal Psycfioloffy (Neiv York W W Norton Company, 
1932), 231-24- 

^Thc Los Angeles City Civil Service Commission has developed tests of 
this kind, for the positions of Auto Fireman, Ambulance Driver, and Motoi 
Truck Driver 

SPifty sixth Annual Report — 1939 And First Half Of 1940, Civil Service 
Commission, City of New York 
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Will probably want to develop their own examinations 
Even the lattei, however, if then expenence with this 
type of test has been limited, may find it helpful to con- 
sider another agency's approach 

Biiildtng the Peifoimance Test 

The initial steps to be followed in constiuctmg the 
peifoimance test do not differ in any important respect 
from those basic to the constiuction of wiitten tests In 
both, the staiting point is caieful analysis of the job for 
which the test is to be designed In constructing a per- 
foimance test the analysis must include actual on-the-job 
observations of both the equipment and the persons doing 
the woik If the test constiuctoi is not himself a compe- 
tent opeiator of the machine, it will not suffice foi him to 
confine himself merely to study of the punted job speci- 
fications and technical liteiatuie and to convcisations with 
expeits and woikeis Such an apptoach is inadequate 
even m the constiuction of wiitten tests, wheie it is all too 
often the lule latliei than the exception, as the only prep- 
aiation to designing a peifoimance test it can produce 
very unfortunate results 

Every one of the foregoing steps — study of the job 
specifications as part of the agency's classification plan, 
study of technical liteiatuie available on the equipment 
and on its operation, and confeieiices with skilled workers, 
supervisois, and acknowledged experts in the field — ^has 
its place in the procedure That place is as a supplement 
to a first-hand acquaintance with the job itself. The pro- 
fessional test constructor must analyze the job sufficiently 
thoroughly to permit himself to identify the skills in- 
volved and to deteimme their lelationship to one another 
and to the whole, and he must discover those individual 
differences which will provide him with essential clues to 
types of test items likely to prove valid in differentiating 
among various levels of performance ability. 

Here it may be worth pointing out that there is per- 
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haps no other phase of the examination piogram in which 
the personnel technician is less likely to tuin out a satis- 
factory job unless he consults with specialists who know 
the practical and technical aspects of the job to be tested 
Both specialists — the personnel technician and the expert 
m the occupational held under consideiation — bring to 
the task certain mfoimation and knowledge of techniques 
which need to be leconciled towaid a common end, that 
of pioduciftg a valid measuring instrument capable of 
fulhllmg the numeious practical consideiations which 
public agencies cannot affoid to forget The test construc- 
toi will want to hnd an expert who knows the job and who 
IS sufficiently piogressive, adaptable, and interested in the 
problems of personnel selection to be coopeiative and 
sympathetic The length of time lequiied to orient such 
a co-workei in the problems of testing will not be great, 
and the effort will pay big dividends in the form of a 
smooth working relationship, a valid measuring instru- 
ment, and a strong ally in the event of later criticism 

It should be kept in mind that the perfoimance test 
should (1) be sufficiently long to include an adequate 
sample of the differentiating essentials of the job, (2) be 
as inexpensive and easy to administer as possible, (3) 
minimize possible differences in achievement resulting 
from lack of immediate familiarity with the particulai 
model of equipment on which it is given, (4) appear 
sufficiently practical and comprehensive to create a favor- 
able impression among those who do not qualify as well 
as among those who do, (5) be capable of uniform admin- 
istration to all candidates, (6) be objectively scored and 
produce quantitative ratings 

Setting the Passing Point 

Since one of the functions of the performance test is 
to eliminate candidates who do not demonstrate adequate 
ability to operate the equipment, passing points must be 
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established with consideiable care Fortunately, this 
problem can usually be approached much more directly 
when performance tests are involved than it can with 
written tests Practical considerations ordinarily make it 
impossible for most jurisdictions to employ standardized 
written tests or to develop satisfactory norms for the tests 
they construct The usual practice, therefore, m agencies 
not bound by restrictive “70 pci cent passing” legislation 
IS to permit such factors as the following to influence the 
location of written test passing points the number of 
examinees, the number of openings likely to occur during 
the life of the register, the general caliber of the compet- 
ing group, whether or not the examination battery in- 
cludes such othei hurdles as a peiformance test or an oral 
interview, and pievious ceitification experiences concern- 
ing the ratio of refusals to acceptances 

In establishing the qualifying point for a performance 
test, on the other hand, the principal criterion must he an 
affirmative answer to the question, “Can the examinee per- 
form the task well enough to meet the employer's mini- 
mum standards?” Production recoids are ordinarily 
available on types of work siilRciently similar to those 
sampled by the test to serve as the basis for setting the 
elimination point Where such records aie not available 
or are not in usable foim, they can generally be obtained 
quite easily, and profitably, too, during the test tech- 
nician’s study of the job 

While such data should usually serve as the principal 
basis for establishing the qualifying grade, they ought not 
to be the sole consideration Some of the other factors, for 
example, which it is frequently important to note are ( 1 ) 
the level of ability of the agency's employees as compared 
with that of other persons doing the same kind of work, 
(2) the immediate, and possible future, condition of the 
labor market In the specific field under consideration and 
in related fields, and (3) the possible effects of nervous- 
ness, atypicality of the test situation, and other factors 
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likely to be present and to lower the validity and reliabil- 
ity of the examination 

Font Perfoimance Tests 

The remainder of this presentation will be devoted to 
describing, with a minimum of discussion, some of the 
forms and piocedures developed in connection with per- 
formance tests for the following foui kinds of jobs Tele- 
phone Opeiator, Graphotype-Addressograph Operator, 
Tabulating Machine Operator, and Duplicating Ma- 
chine Operator 

In consideiing the material, the reader should keep 
the following facts in mmd 

1 Each of the tests was designed for examinees who had "passed” a 
previom hurdle, that of scoring above the 60th percentile in the 
cornbination of then written test score and traming-experiencc 
rating 

2 The haw under which the examining agency operates forbids the 
establishment of any kind of minimum training and experience 
requirements 

3 The operating agency prefers to make no provision for training new 
employees for these positions and requires that a new appointee 
be able to perform the duties of the position almost immediately 

4 Each of the tests was designed to serve as a qualifying examination 
capable of weeding out individuals lacking in sufficient operating 
ability to produce satisfactoiy work, and as a measuring instrument 
capable of producing quantitative ratings of relative ability to per- 
form the work 

5 The validity of none of the tests has been determined through the 
use of statistical procedures For the present, their only claim to 
validity is based on the fact that (1) experts in the fields covered 
by the tests state that they measure what they purport to measure, 
and (2) employees certified from registers established on the basis 
of these tests have proved more uniformly satisfactory to their 
employers than those certified from eligibility lists set up from 
examination batteries which did not include performance tests ® 

of reliability and of (he relation betireen performance teat scores 
and service ratings, vrritten test scores, training scores, and experience scores 
arc now under way 
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The Tesi for Telephone Opoaiors 
This test WRS designed to be iidministered to examinees 
m the 20 counties of the state in which vacancies existed 
for the position of telephone opeiatoi in eithei the mam, 
or a regional, office of a local Boatd of Assistance The 
counties were unusually widely scattered geogiaphically 
The dial system was in use in 60 pei cent of the 20 coun- 
ties and the manual system in the remainder, but the PBX 
boards were all of the cord-type. The switchboards m 90 
per cent of the offices weie connected to four or more 
trunk lines, the laigest two had 12 and IS tiunk lines 
respectively, The number of extension stations varied 
from 11 to 66, but 60 per cent had 20 oi more 

The preliminary siii vey of the physical facilities avail- 
able in each of the 20 counties foi which eligibility regis- 
ters were to be established was accomplished by means of 
a letter in which a questionnaire containing the following 
questions was enclosed 
1. Is your PBX switclibofird of tlie cord type? 

2 Ho\i many city tiuiik lines do you have^ 

3 Is a dial part of the equipmenL of your switchboard? 

4 la the entne clt^ in winch yovir office is iocivtcd equipped with dial 
telephones ? 

5 How many extension stations do you kavc ? 

6 Is there an office neai youi switchboaid in which there are two 
separate extension lines (not two extensions of the same line)? 

7 If so, IS the office in which these two linch are located within heal- 
ing distance of the iingci of a third line? 

8 Would you be willing to permit us to use your switLhboard for 
the peiformancc test If the test is scheduled on a Saturday afternoon 
01 on a week day evening when there are few or no busineiss calls 
likely to tnterfeie? 

Twelve examination centers were established The 
factors which dictated their selection were, (1) the type 
of equipment available m each county for which a regis- 
ter was to be established, (2) the relative proximity of 
counties with similar equipment, (3) the distance each 
examinee would be required to travel, and (4) the cost of 
administration 
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The test itself consisted of a senes of 13 operating 
situations designed to deteimine the examinee’s ability to 
service incoming, outgoing, and extension calls and tians- 
fers of incoming and outgoing calls, all under as nearly 
normal operating conditions as were practically possible 
to achieve The minimum equipment necessaiy foi the 
administration of the test was a cord-type switchboard 
having four trunk lines and five extensions Two forms of 
the test were required one, Form D, for operators of 
PBX installations in communities where the dial system 
was in use, the other, Fojm M, for manually operated 
installations 

The test was admimsteied by two persons (designated, 
in the test, as Mi Albert and Mr Brown), one of whom 
was requiied to be well acquainted with the procedure 
and to have had some practice in its admmistiation The 
two examiners used separate extension telephones but 
were situated within earshot of each other and within 
hearing distance of a third extension telephone. 

The administration, lecordmg, and scoring of the 
test were facilitated by the development of a combined 
“cue sheet” and lating form designed to serve the four- 
fold function of (1) indicating the sequence of opera- 
tions so that each examiner would know what bis task 
was at every stage of the test, (2) listing the phrases to 
be repeated veibatim by both examiner and examinee, 
(3) enumerating the items on which the examinee was to 
be rated, and (4) providing spaces for the examiners' rat- 
ings and comments Each examiner was piovided with a 
copy of this form and, as Mi Albert or Mr Brown, was 
required to originate, maintain, and terminate the calls 
assigned to him and to rate the examinee on each phase of 
every call coming to his attention 

As an aid in orienting the examinee to the test situa- 
tion she"^ was piovided with an Instruction Sheet (Ex- 

^The feminine pronoun la uaed because all etammecs for the telephone 
operator peifomance test y^ere women 
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hibit A) which set forth the geiieial nature of the test she 
was about to take and listed a few simple instiuctions such 
as any experienced opeiator would need to be given on 
starting a new job When the examinee had been per- 
mitted ample time to lead the Instructions, she was as- 
signed to the switchboaid Several minutes, if necessary, 
were then allowed to permit the examinee to familiarize 
herself with any aspects of the board which were strange 
to her and to note the location of the jacks and names 
mentioned in the Instiuctions When the examinee was 
read^y to begin, the receptionist told her to ring Mr 
Biown’s extension and to read hei identification number 
to him from hci admittance slip This operation, as well 
as the first call placed by the examiner, was intended to 
help "break the ice’* and did not entei into the determina- 
tion of the examinee’s scoie, 

Exhibit B IS a reproduction of the test administered 
to examinees lequired to operate switchboaids in dial- 
equipped communities, While the calls comprising the 
manual form of the test were similar in number and com- 
plexity to those included m the dial foim, different cue 
and lating sheets were lequiied because several of the 
opeiations (and, consequently, the points to be rated) 
were not the same for both systems 

Some idea of the variety of realistic operating situa- 
tions existing during the administiation of the test — de- 
spite the fact that all of the calls were originated by only 
two pel sons — may be gatheied from an examination of 
some of the calls the operator is requiied to handle In 
call No 4 (see Exhibit B), the operator connects Mr 
Albert’s extension to a city line A few moments later, the 
operator is telling the peison whose call has come m on a 
trunk line that Mr. Carson’s extension is busy (call No 
5) To maintain the connections required at this stage of 
the test the operator had to put up seven cords In the 
four calls immediately following, the operator was re- 
quired to perform these tasks* 
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Gall No 6 

answer Mr Albcit^s extension, 

transfer tlie incoming call from Mr Carson's extension to Mr 
Albeit’s , 

take down the connection fiom one of the tiunk lines and from 
Ml Albeit’s and Mr Biown's extensions 

Gall No 7 

answer Mr, Biown's extension, 

connect Mr Biown’s extension to a city line so that Mr Brown 
may dial his number tliiough the central exchange 

Gall No 8 

answci an incoming call , 

inform tlie person calling that Mr Brown’s line is busy, 
hold the mconrnng call until Mi Brown’s line is no longer busy 

Gall No 9 

answei Mi Albei t's extension , 

transfer the outgoing call from Mr Brown’s extension to Mr 
Albert’s extension, 

take down the connections fiom one of the trunk lines and 
fiom Mr Albeit’s and Mr Brown’s extensions 

The scoring proceduie was designed (1) to permit 
the immediate elimination of candidates whose perform- 
ance fell below ceitain established minimum standards, 
and (2) to pioduce quantitative ratings reflecting the 
relative operating ability of the examinees who satisfied 
these minimum standards Because both the level of diffi- 
culty of the duties and the relative ability of candidates 
to perform the duties vajied in direct i elation to the si 2 e 
of the county in which the jobs occuired, minimum stand- 
ards (based on the 12 calls compiising the test) were set 
on a class-county basis as follows ' 

Class II Counties 10 completed calls 

Class III Counties 9 completed calls 

Class TV Counties 8 completed calls 

For the put pose of applying the minimum require- 
ments represented by these criteria, a call was considered 
‘‘completed*’ if the opeiation or operations essential to 
recognition of that particular call were earned out sufH- 
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ciently well to leceive ciedit. Thus, an Extension to 
Trunk call was consideied to have been completed d the 
examiiiei had gianted ciedit foi the ungmg stgnalj a 
Tiunk to Busy Extension call if ciedit had been granted 
for the busy )cpottj Transfcriing an Incoming Call if the 
conned} on was maintained t etc 

The actual steps followed in scoring the test are 
enumerated in Exhibit C, which is a reproduction of the 
instructions fuinished the scoieis Two copies of the 
scoung foim (refeiied to in Exhibit C as EB’69S) are 
reproduced as Exhibits D and E The former is the reC' 
ord of an examinee who did not complete a sufficient num- 
bei of calls to qualify m hei county (Class II) . The latter 
represents the lating of an examinee in a Class III County 
who completed considerably moie than the minimum le- 
quiied in her county 

The use of the schedule of ci edits shown in Exhibit F 
made it possible to convert the approximately 7S sub-op- 
eiations comprising the test into quantitative latings.® The 
maximum attainable score foi the test designed for opera’ 
tors of dial equipment was 143, foi operators of manual 
equipment, 123 To facilitate the scoring, keys weie con- 
stiucted which turned the task into a routine operation 
easily perfoimed by cleiks experienced in scoring objec- 
tive tests 


Patt II of tliJs article will appear in the October issue of 
Educauonal and Psychological MuASURcivirNT 


^Ihe correlation between tlic number of crUs completed end the score 
derived by applying the schedule of ciedita shown in Exlubit F is nafntnlly 
quite high In a test compiising 20 or 25 calls it would probably be unnecessaiy 
to go to the added tiouble of weighting and scoring each part of a call How- 
ever, several considerahoriB suggested the dcsti ability of doing so m the par 
tiGular rest described 
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ExlilbU A 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
of the 

DEPARTMENT OF PUBLIC ASSISTANCE 

Harrisburg 

PERrORMANCE TeSI' FOR TfLEPKONC OPERATORS 

SmiEs 1000 
August 1940 

instructions to EXAMINEES 
hnpoitani Failure to follow instructions may 
result in disqualification from the examination 
The examination you arc about to take has been designed to test 
youi ability to perform some of the tasks ordinarily required of a tele- 
phone operator iji the Dcpaitmcnt of Public Assistance 

When voui turn arrives, you will be assigned to a PBX cord-type 
switchboaid The designation strips on tins switchboard will indicate 
the location of four or more ituuk Itnss and the following extension 
stations 

Mr Albeit Mr Carson 

Mr Brown Mr Drake 

OHicial 

You will be given a few minutes to familiarize yourself with any 
aspects of the switchboard which arc strange to you and to note the 
location of each of the j<icks indicated above 

When the Proctor tells you to do so, ring Mr Brown's telephone 
Mr Brown will answer and .isk you to lepcat your Identification Num- 
bci — the number which appcais on your Admittance Slip 

The examination, which consists of making various combinations of 
simple connections, will then begin The first connection you will be 
required to make will be a practice exercise on which you will not be 
graded 

When answeiing calls or acknowledging orders, the following 
phrases must be usedi 

Answering incoming calls — ^'Tublic Assistance " 
Answering extension calls — “Yes, please 
Acknowledging orders — ''Thank you " 

Nate On incoming calls, if the Calling Party requests information 
regarding the Department of Public Assistance or asks to talk to anyone 
besides the four persons whose names are shown on the designation strip, 
the call must be connected to the extension marked “Official/’ 
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Exhibit B 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
of the 

DEPARTMENT OF PUBLIC ASSISTANCE 

HamsbuTg 


Mr Albert □ 

Mr Brown Q 

Peryorm/vnce Tfsi for Telephone Operators 

Series 1000 


1 EXTENSION TO EXTENSION (Practice Exerctse) 


Mr Albert lifts receiver 


Operator answers 

Promptness ( ) "Yes, please?" f ) 

Mr Albert oska for Mr Blown 
"Mr Brown, pIcQsc ” 


Operatoi acknowledges 

"I'liaiik yon" ( ) 

Opct'itoc rings Mr Brown 

BrowuV phone nags ( ) 

Mr Brown answeia ''Mr Brown speiking*' 

ConiKictiDii con^plctcd ( ) 


CoimccUon rniLiitnincd ( ) 

Mr Albert and Mi Brown hang up after a 
few seconds 



2. EXTENSION TO TRUNK 


Mr Drown lifts receiver 


Operator nnaweis 

PromptiiCBS ( ) "Yes, please?" ( > 

Ml Drown qska for city line 

"Clly line, pleisc" _ _ _ _ 


Operator acknowledges 

"Thank you " ( ) 

Operator connects Mr Brown with trunk line 

Enl tone ( ) Promptness ( ) 

Mr Drown dials hated number 

RlnBing signal ( ) 


Mr Albert personally checks number of trunk 

line to which Mr Brown has been connected ( ) 


3 TRUNK TO EXTENSION WPIICH DOES NOT ANSWER 


Mr Brown's call comes in on trunk line 


Operator answers 

Promplntss ( ) 

"Public AssisEsnce *' { ) 

Calling party (Mt Brawn) asks for Mr Cm 
3on ‘*Mr CnrsoUp pleofie '' 


Operator acknowledges 

"Thank you " < ) 

Operator rings Mr Carson 

Car poll's pUoiw rlnga ( ) 

Promptneas ( ) 

Operator gives ringing report Mr Drown 
tells Operator to continue ringing 

TUrtgin^ report every 40 aeconds ( ) 
Appropriate phrase ( } 


Corwcll^ maintained until irana 
fer ( ) 


IdentiUcntion No 


Cenict 

Date 


rorm D 
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Exhibit B (Continued) 

4 EXTENSION TO BUSY EXTENSION, EXTENSION TO TRUNK 


Mr Albert lifts receiver a few seconds after 
Mr Carson^a telephone ffist rings 


Opeiator -inswers 

Promptness ( ) *'YcSj please^'" ( ) 

Mr Albert asks for Mr Brown 

DfOivi^r pJease ” 


Operator gives busy report 

Busy report ( ) PTomptneas ( ) 

Mr Albert asks for city line 
“City line, plcise '* 


Operator -icknowledgcs 

"Tbank you " ( ) 

Operaten connects Mr Albert wjth hunk line 

DjaJ tone ( ) Proinpl^ncsa ( ) 

Mr Albert dials listed number 

Ringing’ signal ( ) 


5 TRUNK TO BUSY EXTENSION 


Mr Albert's call comes in on ttunk line 


Operator answers 

Promptnes*! C ) 

' PiibJjc AssJSlauce ( ) 

Calling party (Mi Albert) iska for Mr Car- 
son '*Hr Carson, pkasc " 


Operator gives busy report and asks calling 
party to hold line 

Busy report ( ) Hold line C ) 

Appropriate phthises ( ) 

Calling party (Mr Albert) hangs irp 



6 TRANSFERRING INCOMING CALL 


Mr Albert lifts receiver 


Operator answers 

Pioniptncsi ( ) "\cs, please?" C ) 

Mr Albert asks to have Mr Cai son's call 
"r et me have Mr Caraou's call, please " 


Operatoi* acknowledges 

Appropriate pbrase ( ) 

Operator transfers infominE call fiom Mr 
Carson to Mr Albert 

Appropriate pbnsc ( ) 

Prinsfcr ( 1 Promptness ( ) 


Connection maiiUatncd ( ) 

Mr Albert and Mr Brown hang up after i 
fe’W seconds 



7 EXTENSION TO TRUNK 


Mr Brown Jifts receiver 


Operatoi answers 

Promptness ( ) '^YeSi please?'* ( ) 

Mr Blown asks for city line 
"City line, please " 


Operatoi acknowledges 

‘Tlrnnk you *' ( ) 

Operator connects Mr Brown with trunk line 

Dnl tone ( ) Promptness ( ) 

Mr Brown dials listed numbci 

Rlnems’ signal ( ) 


Mr Albeit asks Operator numbci of trunk line 

lo wEiicb Mr Brown has been connected ( J 
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Exhibit B (Continued) 

B TRUNK TO BUSY EXTENSION 


Mr Brown's call comca in on trunk line 


Operator answcfs 

Proniplncaa ( ) 

**Puhlic AqBislaiicc ( ) 

Cnlling party (Mr Drown) niks for 

Mr Brown "Mr Drown, plctiae 


Operator gives busy icport and nski calling 
party {Mr Brown) to hold line 

Busy report ( ) Hold Ime ( ) 

Appropriate plirases ( ) 

Calling party (Mr Brown) holds line 

Councelion maintained until Irana 
ler ( ) 


9 TRANSFERRING OUTGOING CALL 


Mr Albert lifts receiver 


Operator answers 

Promptness ( ) ^*YeSp please ( ) 

Mr Albeit asks to have call on Mr Brown's 
line transferred 

"Transfer the call on Mr Brown's line to me please " 


Opciator acknowledges 

Appropnulo pKnec ( ) 

Operator transfers outgoing call from Mr 
Brown's telephone to Mr Albert's telephone 

Approprhlc phnse ( ) 

Mr Albert listens for open line 

Open line ( ) ProinpLness ( ) 

Mr Albert and Mr Blown hang up 



10 EXTENSION TO EXTENSION 


Mr Brown lifts receiver 


Operator answers 

IPromptncas ( ) "Yea please”^" ( ) 

Mr BroAvn asks for Mr Carson 

Cfirsonj please” 


Operator acknowledges 

"Thank you ’’ ( ) 

Operator rings Mr Caraon 

Carson's pUotic rings ( ) 

Mr Corson (Albert) an&wcrs 
'*Mr Carson Spcikinf^ ” 

Connection completed ( ) 


Connection niaintalned ( ) 

Mr Carson (Albert) and Mr Brown hang up 
after a few seconds 



11 EXTENSION TO TRUNK 


Mr Albert lifts receiver 


Operator answers 

Promptpess ( ) "Yc$^ plea'ic”^” ( ) 

Mr Albert asks for city line "City line, please ” 


Operator acknowledges 

"Thank you ” ( ) 

Operator connccla Mr Albert with trunk line 

Uni tftne { ) Promptness ( ) 

Mr Albert dials all but last digit of listed 
mimbcr and holds line 



Mr Brown aaka Operntor number of trunk Une 

to which Mr Albert haa been connected ( ) 
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Exhibit B (Continued) 

12 TRANSFERRING OUTGOING CAIL 


Mr Brovvn lifts receivei 


Operator answers 

Promptnesg ( ) “Yesj please?^' ( ) 

Mr Brown asks to have ctU on Mr Albeit's 
line transfcired 

'^Transfer tlic call on Mr Albert’s hue to nie, please " 


Operator acknowledges 

Appropriate plinse ( ) 

Operator transfers outgoing call from Mr Al- 
bert’s telephone, to Mr Brown’s telephone 

Appropnalc phrase ( ) 

Mr Brown listens for open line 

Open lint ( ) PromplncsH ( ) 

Mr Albert and Mr Diown hang up 



VOICE 

To what eictent la the Operator's voice cleai, distinct, pleasant? 12 3 4 
(1) Very unsatisfactory (2) Unsatistaclory, (3) SntiBfactory, (4) Very satisfactory 

REMARKS 


(8-8 40) 


Examiner 
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Exhibit C 

Procedure for Scoring Telcpfione Operator Performance Test 

Series 1000 

Note ' All scoriQg must be cliecked and must 
carry the initials of both hcorcr and clicclcer 

1 Check to see that theie are two rating sheets and an Admittance 
Slip for each examinee and that the Identification Number on each 
IS identical 

2 Wute the examinee's Identification Number and County in the 
spaces provided on Foim EB-695 (Use Form EB-696 for 
Form M ) 

3 Place a check maik on the lating sheet after the name of each call 
completed by the examinee 

4 Place a check mark in the appropiiatc space on Form EB-695 for 
each completed call and enter the total number of calls completed 
in the box provided 

5 Eliminate fiom fuithci consideration examinees who completed 
fewer than the minimum number of calls required for their County 
(See attached schedule ) 

6 Scoie the rating sheets of examinees who completed a sufficient 
number of calls Place the number of credits after each line and 

place the toUl numbei of credits for each call in a ciicle to the 
right of the last line of the call (See attached schedule of credits,) 

7 Transfer the number of credits for each call to the appropriate space 
on Form EB-695 

8 Place a check maik or an "X” m cacli of the three spaces provided 
aftei the woid “Trunks" on Form EB-695 and refer to the at- 
tached schedule for the number of credits to be entered in the space 
to the right 

9 Place a check mark in the appiopiiate spaces after the woid “Voice" 
on Form EB-695 and icfcr to the attached schedule for the number 
of credits to be entcied in the space to the right 

10 Enter the total numbei of credits earned (Raw Score) in the box 
provided on Form EB-695 
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Exhibit F 

Procedure for Scoring Tclcphonh Operator Performance Test 

Series 1000 

sciirDUiE OF credits 
Foims D and IVL* 

CREDIT-^l POINT CREDIT— 2 POINTS 

Proper plnnsc Ringing extension telephone 

‘'Yes, please?” 

"Public Assistance ” CREDIT— 4 POINTS 

"Thank you ” Busy report 

Promptness (M) Central answers 

Appropriate phrase 
Ringing report 
Request to hold line 

CREDIT-6 POINTS 

E\tension-to-cxtension connection maintained 

Note If connection is completed but not maintained, 3 points 
(D) Ringing signal (after dialing) 

(D) Open line (call 11 only) 

CREDIT— 8 POINTS 

Incoming call transfcried (connection maintained) 

NoU, If transfer is made without maintaining connection, 4 
points 

(M) Outgoing call transferred (open line) 

CREDIT- 10 POINTS 

(D) Outgoing call traniifeired (open line — calls 9 and 12 only) 

USE OF HIGHEST NUMBER TRUNK LINE 
Once (D) 0 (M) 3 

Twice (D) 5 (M) 8 

Thiee times (D) 12 

VOICE 

Satisfactory — 2 (each observer) 

Very satisfactory — 3 (each observer) 

PASSING POINTS 

Class II 10 completed calls 

Class III 9 completed calls 

Class IV 8 completed calls 

in pareaihesis indicates that the credit applies on]y to Form D, 

"M’^ in parenthesis indicates that the credit applies only to Form M 
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R|:CORD 

ARTHUR E TRAXLER 

Ecliic'ihon'il RccdicU Burenu 

AND 

WILLIAM C MC CALL 

University of Soutli Caiolina 

A WELL-ROUNDED guidance program calls for at 
least four types of objective measui es general intel- 
ligence, achievement in various fields of study, aptitudes 
of diffeient types, and interests or motivation Far moie 
progress has been made in the first three of these areas 
than in the fouith In recent yeais, however, there has 
been an especially laige amount of experimentation m the 
last area and some promising measuring instruments are 
beginning to emerge 

The majority of the noteworthy instruments for ap- 
praising interests have been concerned with occupational 
preferences The most important woik in this field has 
been done by Stiong, who has constiucted blanks and pre- 
pared scales for the measurement of the mteiests of men 
with respect to 34 occupations and the interests of women 
in connection with 18 occupations Although the instru- 
ments developed for the measurement of interest in spe- 
cific vocations unquestionably have important guidance 
values, at least two considerations point to a trend away 
from the measurement of interests in occupations as such 
and toward the measurement of interests in broad fields 
One consideration is based on observation and re- 
search. It has been known almost from the first attempts 
to measure vocational mteiests that interests in certain 
vocations are rather highly correlated It has been appar- 
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ent that there are clusteis of occupations that have so 
many points of similauty that interest m one occupation 
IS a stioiig indication of inteiest m seveial others Factor- 
analysis studies have given emphasis to this point, For 
example, by means of a factorial analysis of the Stiong 
Vocational Interest Blank, Thurstone’^ found four interest 
groups These groups were associated with science, lan- 
guage, people, and business 

The second consideration grows out of the practical, 
everyday work of counselors and personnel officers These 
workers have found that fiequentiy when one is attempt- 
ing to guide the development of secondary-school pupils, 
or even of college freshmen, guidance with respect to 
specific occupations is not needed In fact, guidance into 
specialization so caily would in many cases be unwar- 
ranted What IS needed is a valid, reliable measure of 
interests m fauly broad fields so that the individual may 
be guided in the general direction of a gioup of j elated 
occupations, one of which will pel haps be chosen defi- 
nitely when the student has attained greater matunty 
Strong, himself, has been one of the first to recognize 
the need foi broader measurement of interests as well as 
measurement of interests related to specific vocations In 
line with this viewpoint, he has lecently published several 
group scales for the measurement of interests m bioad 
areas 

Certain othei investigators have been working along 
somewhat similar lines Probably the most promising 
new instrument in this general field is the Pieference 
RecoidhyG F Kuder^ 

Desc) ipfion of the Pi efe) ence Recoid 
The Preference Recoid is designed foi use in obtain- 
ing measuies of motivation in the following seven fields 
scientific, computational, musical, aitistic, literal y, social 

^L L Thiirstone, "A Multiple Fiictor Study of Vocationnl Intel eats/’ Per- 
ionnel Journait X (1931) > 198-205 

Frederic Kuder, Preference Recojd (Chicago Science Research Asao 
ciatesj 1939) 
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service, and persuasive It consists of 330 paired-compar- 
ison items of which the following are samples 
A ( 1 ) Drawgiaphs 
(2) Do clerical work 
B ( 1 ) Be a lawyer 

(2) Be a landscape architect 
C 0) Sell insurance 

(2) Do scientific research work 

The subject indicates in each case which one of the pair 
of activities he prefers 

The test is intended for use in high school and college 
It IS administered without time limit The booklet is used 
with separate answer sheets, one for hand scoimg, one for 
machine scoring, and one for self scoring The raw scores 
of an individual student may be plotted on a percentile 
chait and thus a giaphic indication of high points and 
low points with respect to the seven fields may be 
obtained 

Natu\e and Purpose of the Study 
Kuder^ has described the constiuction of the Prefer- 
ence Record in some detail and has reported a consider- 
able amount of statistical data for it Helpful as these 
data are, they naturally do not cover all questions about 
the blank Since no other studies of this new instrument 
were available, it seemed desirable to try to obtain 
answers to certain questions befoie arriving at decisions 
about the use of the blank in a regular testing program. 
The questions which this study attempts to answer are as 
follows , 

1 What IS the letest reliability of the scores on the 
Kuder Preference Record? 

2 Are the scores on the Prefeience Record relatively 
stable over a long period? 

3 What differences are there between the mean scores 
for boys and for girls on the Preference Record? 

F ICuder, “The Stability of Preference Itema," The Jowual of Social 
Psychohffy, X (1939), 41-50 


255 



FDUCAnONAL AND PSYCHOLOGICAL MEASUREMENT 


4 Do the mean scores for different secondary-school 
groups change appreciably with change in grade 
level? 

5 What IS the shape of the mean profiles of umvei- 
sity freshmen in different fields of study? 

The data were obtained by administeimg the Prefer- 
ence Record to fieshmen in the Umveisity of South Caio- 
lina, pupils in Grades 10, 11, and 12 of a high school in 
South Caiolma, and a number of adults who were on the 
staff of an educational oiganization in New York City 

Rehabtlity 

In the manual for the Prefeience Recoid, Kuder 
gives the following reliabilities for the diffeient scales 
scientific, 87; computational, 85, musical, 88, artistic, 
90, literary, 90 ; social service, 84, persuasive, 90, These 
reliabilities weie estimated fiom one administration of 
the test to a group of 84 college students through the 
application of the Kuder-Richardson method of estimat- 
ing reliability coefficients ^ Since the procedure employed 
IS still somewhat experimental and is not as yet generally 
used, it seemed advisable to check the reliability of the 
various scales by the moie famihai test- retest procedure 
Accordingly, 52 college freshmen and 90 high-school 
pupils who had filled out the Pieference Record near the 
beginning of the teim were retested after an interval of a 
few weeks The elapsed time was approximately one 
month for the high-schooI pupils and two months for the 
college students The correlations between the scores re- 
sulting from the two admmistiations are shown m Table 
1 Means and standaid deviations of the distiibiitions are 
also given 

For all scales, the correlations between the two admin- 
istrations of the Preference Record to the secondary- 
school group are above 8 They vary from approximately 
81 to about 91 With the exception of the coi relation 

F Kuder and M W Richardson, "The Theory of the Estimation of 
Test Reliability," Psycfwniilnka, II (1937), 15l-(50 
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TrverE 1 


retest reliability of the KUDER PREFERENCE RECORD BASED 
ON THE SCORES OF SECONDARY SCHOOL PUPILS AND OF COLLEGE 
FRESHMEN IN SOUTH CAROLINA 


Secondary School Piipih | 

1 CoUcKe Preshmen 


N 

V PE Mx 

SDx My 

SDy 

N 

r PE Mx 

SDx 

jii 

SDy 

Scjentific 

■po 

907±013 4i BO 

9 33 41 63 

9 25 

52 

7B?±b3"6 42“OB 

9GB" 

43 65 

9 19 

Couiputa 
lion a 1 

90 

Bl'l+ 024 19 82 

7 24 19 3B 

6 7S 

52 

74B±041 18 IS 

7 86 

18 73 

6 70 

Musical 


876^017 16 69 

7 39 16 24 

7 18 1 

52 

87J-f-02J 18 92 

71i 

J7 S4 

6 49 

Artistic 

90 

857-H019 30 87 

B 97 31 10 

8 04 1 

52 

820+ 031 30 92 

BBG 

29 85 

8 36 

Litdrary 

90 

863± 010 32 30 

9 7i 32E0 10 J4 , 

52 

7B9± 035 31 S6 JO 10 

34 37 

10 67 

Social 




1 






Ser\ ice 

90 

B38±02l 39 97 

9 66 41 20 

10 15 

52 

588+ 061 41 37 

9 43 43 54 

8 02 

PersoiBive 

90 

e3a±021 t5 27 

9 2S 46 27 

8 95 1 

5i 

795± 034 45 62 

9 30 

46 65 

_9 16 


for the social seivice scaicj the conelatioas between the 
two administration^ of the test to the college group are 
above 74 They range upwaid to approximately .87 In 
geneialj these coefficients aie rather high tor coiielations 
based on retesting after an interval of several weeks, In 
fact, most of the corielations seem exceptionally satisfac- 
tory foi a measuiing device that can be administered and 
scored so quickly and that yields as many as seven scoies 
The coiielations based on the secondary-school group 
aie high enough to wan ant considerable use of the Pref- 
erence Record in individual piediction and guidance 
Reliability coefficients above 90 aie theoretically desir- 
able for a test that is to be used in this way, but experience 
indicates that they are very seldom attained in a test that 
yields seveial different scores 

The correlations for the college freshmen tend to be 
lower than those for the secondary-school pupils, 
although there is no significant difference in the case of 
the musical scale The correlation for the social service 
scale, 588) is much the lowest in the group The second- 
aiy-school data indicate, however, that this scale is not 
less reliable than some ot the others ® 

The reason for the somewhat higher correlations at 
the secondaiy-school level than at the college- freshman 
level IS not entirely clear It was thought at first that pos- 

®8irice this correlation was ant of lirip with the others, it wqb recheoketl with 
the greatest of care Everv paper was rescored, the data were redistributed, 
and the entire calculation was carried through a second tune It seems certain, 
therefore, that no eiror of a clerical natiiie is involved 
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sibly the combining of the three high-school grades into 
one gioup had inci eased the variability and thus raised 
the corielations Howevei, a comparison of the standard 
deviations shows that in general the variability is not 
greater foi the high-school gioup 

The diffeience in magnitude of the coiielations may 
be due simply to the longer time inteival between admin- 
istrations of the Piefeience Recoid to the college group 
By the same leasonmg, the coi relations foi the high- 
school pupils may be a little lower than they would have 
been if only a few days had elapsed befoie the test was 
repeated It is impiobable, howevei, that the basic inter- 
ests and motives of either high-school oi college students 
change significantly duiing a peiiod of a few weeks 
Moreover, the lepetitioii of the Piefeience Recoid after 
a veiy biief peiiod would have been subject to the limita- 
tion that a inemoiy factoi might have pioduced spunously 
high corielations 

The retest conelations based on the secondaiy-school 
group couespond rathei well with the estimated reliabili- 
ties leported by Kudci The coii elation obtained in this 
study for the scientific scale is a little highei than Kudei’s 
figure. The two sets of i eh abilities for the musical scale 
and the social seivice scale would agree exactly if those 
repoited heie weie lounded to two decimal places. In 
the case of the other four scales, the retest conelations foi 
the secondary-school pupils are lower than Kuder's reli- 
abilities, but the diffeiences are not maiked 

The college-freshman retest coi relation for the musi- 
cal scale is in very close agreement with Kuder’s reliabil- 
ity coefficient. The college-freshman conelations for the 
other scales aie significantly lower than those found by 
Kudei, but the only sti iking difference is between the 
conelations foi the social service scale 

Means and Standard Deviations 
Although this is not concerned with one of the main 
questions raised in this study, it may be noted in passing 
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that the means and the variabilities of the distributions 
lesulting from the two admmistiations of the Preference 
Recoid tend to be closely similar m both gioups Appar- 
ently the practice effect was negligible > that is, the scores 
did not tend to be higher on the second administration as 
a lesult of the subjects’ having taken the test pieviously 
The absence of evidence of piactice effect is a further 
point in favoi of the Pieference Recoid 

Anothei interesting observation based on the means 
and standaid deviations shown m Table 1 is that, on the 
whole, the diffeience between the two groups m cential 
tendency and vai lability aie slight This observation sug- 
gests that interests in the seven areas involved aie rela- 
tively mature by the time pupils enter the secondary 
school The laigest difference in favoi of the college- 
freshman group IS found in the social seivice scale, a 
result which famihaiity with scoies of high-school and 
college students on the Strong Vocational Interest Blank 
would lead one to expect 

Stability of the Scoies 

We have just noted that the retest couelations for the 
scales of the Kuder Prefeience Record tend to be rather 
high foi an interval of a few weeks But how high would 
they be for a rathei long peiiod — let us say, a year or 
more? In other words, what is the stability of the scores 
and what is their value for long-time predictions? Some 
information relative to these questions is provided by the 
correlations in Table 2, which are based on the retesting 

TABLE 3 

CORRELATIONS BETWEEN SCORES MADE DY SIXTEEN ADULTS ON 
TWO ADMINISTRATIONS OF THE KUDER PREFERENCE RECORD 
AFTER AN INTERVAL OP APPROXIMATELY FIFTEEN MONTHS 


Scale 

N 

r PE 

Mk 

SDx 

My 

SPy 

Scientific 

16 

S28T053 

47 25 

12 1+ 

+7 50 

10 9+ 

Computatmnal 

16 

864+ 0+3 

24 S6 

8 27 

23 81 

3 68 

Musical 

16 

933:h 022 

22 00 

8 46 

22 63 

8 31 

Artistic 

16 

698+086 

31 38 

6 90 

31 25 

B35 

Literaij 

16 

SIOT ois 

4+35 

3 43 

44 3S 

8 13 

Social Service 

16 

611+ 106 

42 38 

717 

41 SO 

3 29 

Persuasive 

16 

883+ 037 

31 38 

11 70 

32 88 

12 42 
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of 16 adults With the Preference Record after an interval 
of appioximately IS months 

The coi relations in Table 2 lange fiom above 6 for 
the social service scale to above 9 foi the musical scale 
The conelations foi all the scales e:xcept the aitistic and 
social SCI vice ones are above 8 The leliability of these 
coi relations is of course limited by the small number of 
cases For example, the con elation coefficient for the 
artistic scale was loweied consideiably by a marked 
change in the score of one person 

Nevertheless, the correlations in Table 2 suggest that 
in geneial the scoies on the Prefeience Recoid are rather 
stable foi a period as long as IS months and that they 
provide a fauly satisfactory basis for long-time piedic- 
tions, Emphasis is given to this point when one examines 
the piefeience profiles of the different individuals In 
neaily all cases, the high and the low points lesultmg 
from the first administiation of the test weie closely simi- 
lar to those based on the second administiation, The pro- 
files for two individuals, in terms of peicentiles, are 
shown in Figuies 1 and 2 

Sex Dlffeiences 

When one is iiiteipieting the profile of an individual 
on the Pieference Record, it is of some interest to know 
whether there aie charactenstic diffeiences between the 
average scores of boys and gals The mean scoies made 
on the vaiious scales of the Pieference Record by groups 

TADLE 3 


MEAN SCORES OF GROUPS OF BOYS AND GIRLS IN HIGH SCHOOL 
AND IN COLLEGE ON THE KUDER PREFERENCE RECORD 


Scale 

Hieh 

School 

Doya 

High 

School 

Gills 

Freshman 
Boys 
in a State 
University 

Freshman 
Girls 
m a State 
University 

Freshmen 
in '1. 

Girls’ 

College 

Number of Cases 

152 

135 


173 

SB-l 

Scientific 

48 1 

37 9 

47 9 

41 4 

41 7 

Computational 

21 6 

16 9 

21 8 

17 7 

17 8 

Muaicftl 

U2 

19 2 

IS 8 

19 7 


Aitistic 

27 4 

29 9 

26 9 

321 

29 7 

Literary 


33 4 

32 8 

367 

35 8 

Social Service 

37 9 

44 8 

40 7 

45 4 

46 7 

Persuasive 

47 9 

44 9 

49 4 

42 8 
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1 Scientific 

2 Computational 

3 Musical 

4 Artistic 


5 Literary 

6 Social Service 

7 Persuasive 


1 3 3 4 5 6 7 



Figure 1 Profile of a Giil Secretary Wirh a 
Eong-Sranding Intel est in Music 
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1 Scientific 5 Liteiary 

2 Conipulatioiial 6 Social Seivice 

3 Musical 7 Persuasive 

4 Aitistic 



Figure 2 Profile of a Machine Scoring Supeiviaoi, an OccupaLion 
far which Interest and Ability in Computation Are Very Important 
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of high-scfiool boys and girls and college-freshman boys 
and girls in South Carolina are piesented in Table 3 
As one might expect, m both groups the boys aie on 
the average higher than the girls in scientific, computa- 
tional, and persuasive preferences, while the giils surpass 
the boys in musical, aitistic, literary, and social service 
preferences The largest diffeiences are for the scientific 
scale However, even the smallest differences in medians 
amount to neatly 10 percentile points It appears, there- 
foie, that sex differences m preferences should be taken 
into consideration when the scores on this test are 
interpreted 


Giade-Level Differences 

When the results of achievement tests are being stud- 
ied, the usual procedure is to inteipiet the scores m terms 
of the noims for the grade the pupils are in Is it neces- 
sary to follow this piocedure with the Pieference Record 
or are the results in different grades so similar that one is 
justified in disregaiding grade level, at least as far as the 
secondary school is concerned? Some information on this 
question is given in Table 4, which shows mean scores of 
boys and of girls m Grades IX, X, and XI of a South 
Carolina High School 


taiile 4 

MEAN SCORES MADE ON THE PREFERENCE RECORD BY GROUPS OF 
BOYS AND GIRLS IN GRADES IX, X, AND XI OF A SOUTH CAROLINA 

HIGH SCHOOL 


Scale 





Girls 


Grade 

IX 

Grade 

X 

Grade 

XI 

Grade 

IX 

Grade 

X 

Grade 

XI 

Number of Cases 

48 

77 

37 

42 

72 

21 

ScieiitJfic 

4B8 

48 4 

45 8 

36 1 

390 

38 0 

ComputQtionjil 

20 5 

32 0 

22 3 

16 9 

16 9 

16 7 

Musjcal 

11 3 

13 6 

15 2 

18 9 

174 

19 8 

Artistic 

371 

281 

25 9 

313 

29 4 

28 8 

Literary 

30 3 

30 5 

30 1 

32 9 

33 9 

33 9 

Sod'll Service 

40 0 

37 0 

36 3 

44 0 

446 

47 1 

Persuasive 

47 8 

47 5 

49 2 

42 3 

46 0 

46.4 


Because of the lather small number of cases, the means 
are not highly reliable indicators of the preferences at the 
different grade levels. Nevertheless, the fluctuations in 
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mean scoies made by the pupils in the thiee grades are not 
great Moi cover, theie is no consistent trend towaid the 
obtaining of highei or lowei ratings with advancement 
in grade level The data in Table 4- are by no means con- 
clusive, but, asfai as they may be inteipieted, they suggest 
that diffeient norms for these thice grades aie not needed 

Mean Piofihs fo) Different Fields of Study 
The means and standard deviations of scores of Uni- 
versity of South Carolina freshmen classified according 
to field of specialization aie shown in Table 5 The per- 
centile ratings of the mean scores aie indicated graph- 
ically m Figure 3 

rADl£ S 

MEANS AND STANDARD DEVIATIONS OF SCORES MADE ON KUDFR 
PREFERENCE RECORD BY VARIOUS GROUPS OF FRESHMEN IN THE 
UNIVERSITY OF SOUlH CAROLINA, INCLUDING BOTH MEN AND 

WOMEN 



LiiBinfcniiffdJ S ) 

Journilism 



Alt 



Erlucallon(A B ) 

Scale 

N 

SD 

N 

Mean 

b 

U 

N 

Mean 

SD 

N 


S D 

ScIcnMne 

79 


30 

^’20 

26 

39 

23 

b 

10 

15 

41 

93 

7 

19 

36 

39 

50 

10 06 

CotnpufQlioin] 

79 

34 

29 

S 32 

7fr 

11 

92 

s 

48 

15 

17 

b7 

iO 

S2 

36 

16 

B3 

6 66 

Music'll 

?9 

14 

44 

6 4S 

26 

18 

77 

6 

11 

15 

18 

07 

5 

31 

36 

18 

72 

7 85 

ArbiaVic 

79 

i9 

la 

7 17 

26 

2S 

23 

7 

22 

IS 

45 

67 

7 

U 

36 

2S 

so 

7 96 

Literary 

79 


3a 

7 44 

26 

54 

(d 

6 

S5 

IS 

31 

13 

S 

54 

36 

36 

2B 

8 69 

Sociftl Service 

79 

-JO 

14 

7 21 

'»6 

41 

00 

9 

38 

15 

44 

87 

i 

OB 

36 

44 

7B 

11 46 

Peraiisislvc 

. 79 

46 

90 

BOd 

26 

18 

IS 

9 

33 

IS 

42 

73 

8 

79 

16 

44 

28 

9 48 


Coinnterccdl 5 ) Sccrctnnal Science Pre Medicine rilariuacy 
Senic N Menu S U N Mean S D N Mean 5 D N iMean S D 


ScEcntIfia 

83 

10 

flS 

B 

50 

63 

36 

59 

8 

03 

53 

56 

24 

8 

66 

13 

54 

69 

7 80 

Comnutfillonnl 

MuslcaL 

82 

21 

71 

6 

47 

83 

20 

B6 

7 

n 

53 

17 

04 

5 

3\ 

13 

16 

54 

5 21 

82 

]5 

19 

6 

47 

63 

19 

12 

5 

89 

53 

16 

61 

7 

22 

13 

16 

54 

R 16 

AiilalU 

82 

24 

06 

5 

16 

63 

30 

16 

9 

LI 

S3 

2S 

77 

7 

fl2 

U 

29 

99 

6 62 

Lllerary 

62 

31 

12 

B 

S3 

S3 

33 

B4 

8 

55 

53 

34 

26 

7 


13 

JO 

08 

7 13 

Social Service 

82 

40 

85 

7 

79 

63 

43 

00 

0 

09 

53 

48 

59 

7 

25 

13 

12 

65 

Win 

Persuasive 

B2 

S3 

83 

H 

66 

63 

45 

17 

9 


S3 

14 

62 

10 

55 

11 

43 

62 

8 39 


Arts and Arts 'incl 

Pre Law Sc«:nces(A D ) Scit.nccg(B S ) 


Scale 


SD 

W MeRik 

SD 

N 

Mean 

SD 

SuEnlitic 

32 

40 

00 

3 

50 

169 

41 

13 

B 

60 

49 

51 

45 

9 

41 

ComnutaUaAal 

32 

20 

69 

5 

85 

169 

17 

6-1 

6 

6S 

49 

19 

57 

6 

9B 

32 

16 

69 

5 

22 

169 

19 

U 

7 

32 

49 

16 

18 

7 

17 

Arli^lic 

J2 

24 

13 

7 

38 

169 

30 

66 

9 

30 

49 

27 

61 

8 

32 

Literary 

32 

38 

00 

3 

65 

170 

35 

71 

10 

21 

49 

34 

O 

a 

59 

Socini Service 

32 

38 

13 

S 

34 

169 

44 

33 

10 

3B 

49 

43 

12 

10 

07 

rersiiaslve 

32 

a? 

94 

10 

99 

170 

44 

2^ 

10 

42 

49 

44 

10 

iO 

05 


The piofiles of no two groups aie alike, The greatest 
similarity probably is in the profiles for the pre^medical 
and pharmacy groups, but even here the coirespondence 
is not especially close when all the scores are considered 
Most of the high points and low points in the profiles 
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Percentiles 


Percentiles 
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Figure 3 Mean PruJiJes of Groups of Freshmto Who Hove Indicated 
Educational or Occupational Choices in the Fields Named 
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occur accoiding to one’s expectation The art gioup, for 
example, is lathei low in the scientific scale but very high 
in the aitistic scale. The journalism gioup is close to the 
thirtieth peicentile on the scientific and computational 
scales, but almost up to the ninetieth percentile on the 
liteiaiy scale The commerce gioup is approximately at 
the eightieth percentile m computational and persuasive 
interests, but close to the thiity-fifth percentile in scien- 
tific and hteiary inleiests Both the pre-medical and the 
pharmacy students are high in scientific interests The 
pre-medical students are also faiily high in social seivice 
inteiests. The pie-law students ale close to the thirtieth 
percentile on the scientific and social service scales, but 
near the eighty-fifth peicentile on the persuasive scale 
and not far below the seventieth peicentile on the compu- 
tational scale The engineering gioup is above the sev- 
entieth percentile in computational and scientific inter- 
ests, but below the thii tieth percentile m literary interests 

In the manual foi the Piefeience Recoid, Kuder has 
given median profiles for groups of students who have 
chosen occupations in the fields of writing, social seivice, 
physical sciences, political science, business and account- 
ing, veterinary medicine, medicine, and law A compari- 
son of the piofiles in Figure 3 with Kudei’s profiles re- 
veals notewQithy similaiities between those for (1) jour- 
nalism and writing, (2) commeice and business and 
accounting, (3) pie-medical course and medicine, (4) 
pre-law and law, and (5) aits and sciences (B S ) and 
physical sciences The fact that the profiles deiived from 
two independent sources for groups in the same general 
areas are similai and are on the whole in agreement with 
what one would reasonably expect is favorable to the 
reliability and validity of the Preference Recoid 

Conclusions 

1. The retest reliability of the scales of the Kuder 
Preference Record is rather high The correlations be- 
tween the scores resulting from two administrations of the 
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Prefeience Record to a gioup of high-school pupils with 
a time interval of about one month weie above 8 for all 
seven scales The corielations between the scores based on 
two administrations of the Record to a gioup of college 
freshmen with a time inteival of two months were above 
7 for six of the seven scales 

Z The scoies on the Preference Recoid do not seem 
to be influenced by piactice in taking the Recoid when 
theie IS an inteival of several weeks between administra- 
tions of the recoid The mean scores resultJng from the 
second administration of the Record were not appreciably 
or consistently higher than the scoies obtained the first 
time the Record was taken, 

3 The scoies on the Prefeience Record appear to 
have consideiable value foi relatively long-time predic- 
tions as fai as adults are concerned The correlations 
between the scoies of 16 adults after an interval of IS 
months were faiily high, varying from about 6 to slightly 
above 9 

4 There aie notewoithy sex differences between the 
mean scores of high-school and college boys and girls 
On the average, the boys exceed the girls in scientific, 
computational, and peisuasive preferences; the girls are 
higher than the boys m musical, aitistic, literary, and 
social service preferences 

5 It appears that mteiests and motivation in the 
seven aieas involved are relatively mature by the time 
pupils leach the secondary school The differences be- 
tween the mean scores of pupils in Grades IX, X, and XI 
were found to be slight, and theie was no consistent trend 
towaid highei scores with inciease in grade level Simi- 
larly, the differences between the mean scores of the sec- 
ondary-school and college freshman groups were small 

6 Mean profiles were found for 11 groups of univer- 
sity freshmen classified according to field of study or 
occupational choice The profiles tended to have high 
points and low points at the places where one would 
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expect them to be A comparison with piofiles found by 
Kudei showed that those foi groups in the same geneial 
fields were sanilai in shape 

In general, the data in this article are favorable to 
the Kuder Prefeience Recoid, By far the most important 
question that remains to be answeied has to do with the 
validity of the Record, Do the scales really measure what 
they purport to measuie? While certain aspects of the 
data reported in Kuder’s manual and in this aiticle imply 
consideiable validity, theie is at present little direct evi- 
dence concerning the validity of the Preference Record 
One of the wnteis reported a small amount of data on the 
validity of the Record in Buros’ 1940 Mental Measuie- 
ments Y eat book, but there was nothing conclusive in the 
findings Fiiither study of the Preference Record could 
well be directed toward this question 
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THE RELIABILITY OF RATIO SCORES 


LEE J CRONBACH 
State College of Washington 


E ducational measurements often give rise to 
quotients or ratios obtained when one score is divided 
by another The intelligence quotient, achievement quo- 
tient, and per cent accuracy scores aie examples For 
the effective interpretation of such a measure, it is im- 
portant that an appropriate estinnate of its reliability be 
obtained While a formula for the reliability of ratios 
has been presented by Hohmger, this, like other ap- 
proaches, has limitations which apparently have not pre- 
viously been discussed The present article is intended 
to summarize the procedures which may be applied to 
ratio scores and to indicate the conditions under which 
each is appropriate 

Ratio scores appear to be particularly important m 
dealing with certain new-type tests such as those now 
being published by the Progressive Education Associa- 
tion In Test I 41, Social Problems,^ for example, the 
student is presented with a description of a social prob- 
lem, asked to state which of several solutions he favors, 
and then to check, from a long list, which reasons he 
would advance to support his decision Since the student 
may check as many reasons as he wishes, there is m prac- 
tice a wide range of ‘‘Total Reasons^’ among students 
One important datum is the extent to which the student 
checks reasons which are inconsistent with his conclusion 


1 Socutl Pi^oblemj Chicago Progreasirc Education Aasociaiion 
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and really support one of the other solutions This is 
expressed in the score “Number Inconsistent ” In order 
to compare one student with anothei, it is convenient to 
eliminate the comprehensiveness factor > expressing his 
performance m a “Per Cent Inconsistent” score by divid- 
ing Number Inconsistent by Total Reasons 

Relesf method One of the most satisfactory estimates 
of the reliability of such a score is to be obtained by the 
retest method This method has generally been used to 
deteimme the reliability of the IQ It is not easy to 
rule out possible practice effect, even when paiallel foims 
aie used For some tests it is difficult to prepare a paiallel 
form, even where two forms aie available, it is often 
desired to estimate the reliability without the tiouble a 
second testing requires Once data fiom two forms are 
available, it is a simple matter to compute the two ratios 
for each student and correlate them It will be shown 
below, however, that a coefficient so obtained may not be 
equally appiopiiate for all scores m a given population. 

Kudet-Riclmdson method Where retesting is im- 
piacticable, it is customary with ordinary tests to use the 
Spearman-Brown split-half procedure or the recently de- 
veloped Kudei -Richardson method The Kuder-Rich- 
ardson method is based upon a summation of the variance 
of the items composing the total score,® since a ratio score 
cannot be conceived as composed of a sum of items, the 
method is not applicable (except in that special case 
where the denominator of the ratio is a constant for all 
students) Whether a modification of this method can be 
developed which is appropriate for ratios is not known 

Split-half method The Spearman-Biown formula 
is based on the assumption that the two halves into which 
the test has been split may be added to form the whole 
In the case of ratio scores, this assumption does not hold 
Since the denominator of the ratio is, in general, different 

E Kuder and M W Richardson, “The Theory of the Eatlmntion of 
Test Reliability," Psychometnka, II (1937), 151-60 
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in each half, 4^1 + -^ is not equal to or 1/2 ~ It 


1/2 


2/2 


would be possible to correlate-^^ with-^^, this would, 

b b 


by the Speai man - Brown formula^ yield Since 

this disregards the possibility of erior m measuring the 
denominatoi, it does not estimate the reliability of the 
ratio in the usual sense of It is obvious that these 

objections do not apply to the special case where the de- 
nominator IS the same for every student, as in the per cent 
accuracy score on a test where every student attempts 
every item Heie, the reliability of the ratio is the same 
as the reliability of the numerator 

Gomputation by formula. Statistical formulas for 
obtaining the mean, standard deviation, and correlations 
involving ratios by indirect methods were developed by 
early workers These formulas, obtained by assuming 
that the variation of the denominator is small compared 
to Its mean, are as follows 


li and so on, then 

U d a 
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If the reliability of a score is conceived as its correlation 
with Itself, one may substitute a for c and b for d m 


®G U Yule qnd M G Kendall, Introduction to the Theory of StaiiiUes 
(12th ed , revised, Philadelphia Lippincott, 1937)t pp 299 300 

*Karl Pearson, "On a Form of Spurious Correlations Which May Arise 
When Indices Are UBCd m the Measure of Organs/' Proceeding! of the 
Society, LX (1897), pp 489 ff 
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formula (3)^ obtainmg this formula for the reliability 

't aa ^ ati^a ai^a^'b ~b ^ 

Va 2 1 ay’^aV'b ”b 

This foimula may be employed wherever the necessary 
data can be computed Since Uaj ibh, and the other vari- 
ables can be obtained without a letest, data fiom a single 
testing are sufficient Either the split-half or Kuder- 
Richaidson method may be used to estimate these relia- 
bilities It must be emphasized, however, that the 
formula is applicable only when the basic assumption is 
valid, namely, when the spiead of the denominator vari- 
able IS small compaied to its mean 

In actual school testing of a single grade, the variation 
m mental age or chronological age is noimally quite a 

small fraction of the mean M A. or C A The ratio ^ 

for M,A. and C A is likely to be sufficiently small that 
higher powers can be neglected , as a result, the formula 
can be applied to cithei the A Q or the I Q. under this 
condition An empirical test of the formula by Morley 
showed close correspondence between foimula results and 
results from a retest of 381 pupils, all in Grade VIA, on 
several achievement quotients “ It does not follow that 
the formula can be applied to the achievement quotient 
if several grades are included in one population 

When the Pei Cent Inconsistent scoie is studied, one 
linds that the assumption does not necessaiily hold. The 
average student checks less than 50 leasons, but the range 
in Total Reasons often is from 10 reasons to 70 reasons 
The coefficient of variation of the denominator in such a 
case IS so high that error may follow when the formula is 
applied. A further confusion lies m the fact that all 

i^This farmuln Hral developed by Holzingef See K J Holzinger, 
"Formulas for the Correlation between Ration/' Jountal of Ei^ucflUonal 
Psychology, XIV (lS>23)j 344-47 It may nlso be derived directly by approvima- 
tion from, expansions of Infinite senes 

®C A Morley, "Ihe Reliability of the Achleverncnt Quotient," Joumal of 
EducaUonal Psychology, XXI (1930^ 355-56 
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scores m a given population aie not equally reliable A 
digression to demonstiate this point is necessaiy befoie 
inethods of attacking this problem can be presented 

It IS well known that the leliability of a test is a func- 
tion of the length of the test The student who marks 
three inconsistent reasons out of 10 reasons used, and the 
student who marks 30 inconsistent reasons out of the 100 
he uses, both receive Per Cent Inconsistent scores of 30 
The score of the lattei student is an estimate of his in- 
consistency based on 100 responses , the estimate of the 
formei student is based on only 10 If the former student 
weie to maik one additional reason, his Per Cent Incon- 
sistency score would inciease to 36 per cent or decrease 
lo 27 pei cent, if the second student were to mark one 
more reason, his scoie would shift upward only to 30 7 
per cent oi dowtiwaid to 29 7 per cent, depending, of 
course, on whether the additional reason were consistent 
or not Similarly, a change of only one point in the 
numerator pioduces a much greater change in the ratio 
for the student whose denominator score is low From a 
logical point of view, then, we would expect the standard 
eiroi of measurement of a ratio score to increase as the 
denominator decreases The standard enor of measure- 
ment and Che reliability coefficient are inversely related; 
therefore, the reliability of a score increases as the size 
of the denommatoi increases 

Possibly reference to the per cent accuracy concept 
will furthei clarify this point If we were to ask a stu- 
dent a single question, he could answer it correctly or 
inconectly or could omit it If we desired to know the 
percentage of his attempts that were successful, we could 
compute a per cent accuiacy score, which, based on one 
question, could only be 100, 0, or indeterminate Certainly 
no measure of this sort, based on one item, would be con- 
sidered significant If two questions were asked, he could 
have both right, both wrong, one right and one wrong, or 
could omit either, or both His per cent accuracy score, 
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under each of these conditions in oidei, would be 100, 0, 
50, 100 or 0, 01 irideteiminate Obviously, when a score 
of 50 per cent is possible, disci imination is liner than 
when only scores of 100 and 0 are possible Similaily, as 
the numbei of items aUempted inci eases, discrimination 
becomes incieasmgly fine, which means that accuracy, 
hence leliability, of measuiemeiit increases No mattei 
how many items are added to the test, if a student omits 
all the items no meaningful pei cent accuracy score can 
be obtained, and, in geneial, the accuracy with which his 
perfoimance is measuied depends upon the number of 
items he attempts Tivnible has suggested"'^ that this 
applies not only to the tatio scoies, but to scores on any 
test wheie the student is instructed to lespond to as many 
Items as he wishes, this pattern is found in seveiaJ tests 
of the Progiessive Education Association senes 

Since It has therefore been demonstiated that the re- 
liability of any scoic is a function of the size of b, as 

b 

well as of the test used and the gioup measuied, one may 
raise the question: how can a foimula foi reliability of a 
ratio, giving a single answei, be meaningful? The answer 
may be obtained by recalling the basic assumption under 
which the formula was obtained, to wit, that it holds only 
for those cases wheie cr„ is small compaied to h This is of 
coursemostlikely where either (a) there is little variation 
in b scores withm the gioup or (b) values of b are high In 
the formei case, all scores will have about the same re- 
liability, which can be estimated by formula (4) In the 
latter case, the value obtained by the formula is a limiting 
value It was pointed out that the leliability of a ratio 
increases as the denominator inci eases, other things being 
equal Since, as the denominator increases, the case is 
approached where powers of On aie completely negligible, 

b 

It follows that the value given by foimula (4) is valid 

lln correspondence 'with the writer 
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only where the assumption is met, and that for lower 
values of b one may expect a lower reliability 

Another approach to the same type of statistic gives 
a formula foi the standaid eiror of measurement If a 
large number of measures of a, say Hj, , a^, and 

coi responding measuies Z> 2 , j are obtained 

for the same person by a senes of n measuiements, a set of 
values will also be obtained, The standard deviation of 
this set IS by definition the standard error of measurement 
of the ratio From (2), 


~ + t>j?) (S) 

If one assumes that errors in a are independent ot errors 
jn hi when the same person is tested repeatedly, 



or, if s IS used as a symbol for the standard error of meas- 
urement, 



where and can be obtained by the usual methods This 

reduces, using the identity j = o V 1 - — r<i, to (4) It 
must again be stressed that this formula is valid only 
where Oj, is small enough that powers may be disregarded 

T 

Since the formulas (4) and (7) are valid for some 
sets of scores and invalid for others, some procedure must 
be developed to determine where the formula is appli- 
cable A useful test to determine whether the formula 
applies to any set of scores is to obtain values for Mt and (T^ 
empirically If the values are close to those found by 
formulas (1) and (2), the assumption may be considered 
reasonable in this case; if discrepancy appears, the 
formula should not be used 
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With such a score as Per Cent Inconsistent on 1 41, the 
assumption may hold foi high values of b, but not for 
scores based on small values of b (Total Reasons) In 
this case it is possible to compute by formula the relia- 
bility of those scoies wheie the assumption applies, but 
not for cases wheie the dcnominatoi is small To de- 
termine the range wheie the foimula can be used, the 
following procedure has been found efficient. 

I A scattei diagiflm of a against b is made for the sample 

2, An estimate is made that the assumption will hold for values of b 
gieater than a certain value, say b' For all cases where b is equal 
to or gicatei than b', tlie standard deviation and mean of b, and 
aic computed from the appropiiatc lows m the scatter diagram 

3 In a separate scatter diagiam, i is plotted agamst bj usmg the same 
class intervals for b as before For all of the cases which fall in iqws 
so tliat b ^ b'j cri and Mi are computed by the usual method 

4 Using formulas (1) and (2), m and Me are computed from the data 
obtained m step (2) If these values aic equal, or viitually so, to 
those obtained empirically foi the same population m step (3), the 
assumption that the vaiiation of b is small compared to b itself is 
piobably justified for b ^ b' 

5 If the values from steps (3) and (4) aie equal, it is possible that the 

assumption holds for a value b' If the values fiom steps (3) 

and (4) aie not equal, it is necessary to test a value b'^ ^ b' In 
either case, a new hypothesis is made, that the assumption holds if 
b ^ b" Using the same scatter diagiaras, values tue calculated for 
the means, standard deviations, and lai for cases at or above i" This 
IS a compaiativclv simple step, as most of the previous computation can 
be used again Again, the values of Mi and iri obtained by foimula 
are checked against those obtained empirically By a repetition of 
this process, it is possible to dctcimine tlic smallest value b^^^^ of b for 
which the statistics derived empirically and by the formula are equal 
within prescribed limits of accuracy. It is piobably unnecessary to 
compare the means, as a check between the estimated and empirical 
standard deviations should be an adequate test, since it requires little 
additional work to check means also, it is probably wise to do so 

276 



RELIADILITY OF RATIO SCORES 


Having identified the lange of b for which the as- 
sumption holds, one may compute the reliability oi 
standard eiror of measurement by formula (4) or (7) 
Except for Uh, the statistics which enter this 

equation have already been computed in the steps above. 
In many cases it is most simple to compute and ij,j, by 
the split-half method If the Kuder-Richardson method 
IS used, It IS necessary to make an item analysis of those 
papers whose ^-values are sufficiently high, separate from 
such an item analysis of all papers as would oidinarily be 
made for othei purposes It is possible to plan the item 
analysis m advance, ranking papeis in the ordei of their 
i-scores, so that the Kuder-Richardson method may be 
applied to a portion of the population economically 

Siimmaiy 

Methods appropiiate foi computing the reliability of 
ratio scores have been discussed They aie 

(1) The retest method, which requires construction 
of a parallel form for greatest meaning The necessity of 
a second testing makes this inapplicable in many situa- 
tions. A coefficient so obtained assumes that the relia- 
bility of all scores m a group is the same 

(2) The Spearman-Brown foimula, applied to the 
correlation between scores based on a splitting of the test 
into two parts This is generally invalid for ratio scores. 

(3) The Kuder-Richardson formula, which may be 
used only where the denominator of the ratio is a con- 
stant 

(4) The Holzinger formula foi the leliability of 
ratios, valid only if the variation of scores in the denomi- 
nator IS small compared to the mean of the denominator 
for the group A related formula for the standard error 
of measurement developed in this paper is valid under 
the same conditions 

It was pointed out that ratio scores within the same 
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population, and even latios which are equal in size, may 
not have the same reliability. The standard error of 
measurement inci eases as the denominator deci eases It 
follows that a single leliability coefficient foi a latio score 
IS not meaningful, except for data wheic the variation m 
the denominator is small compared to the denominator 
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GUIDING STUDENTS TO BECOME 
SELF-GUIDING 


JOSEPH S KOPAS 

Fenn College 

T O adjust oneself to modern living requires a degree 
of personal development that informal, hit-and-miss 
efforts of the individual do not supply Too many people 
are frustrated and unhappy because their preparation for 
adjusting themselves to oui complex society has been only 
incidental In this day and age a person requires train^ 
mg specifically diiected at teaching him how to get the 
most out of life and the most out of himself He must 
learn to appraise himself, to direct his personal develop- 
ment, to take advantage of opportunities foi growth, and 
to evaluate his progress from time to time The devel- 
opment of these skills should not be left to chance They 
are far too important for that An organized, formal 
program of turning in self-guidance is needed 

In recognition of this need, a guidance program has 
been developed over a period of years at Fenn College 
The use of evaluative procedures is an integral pait of 
the program 

Fenn College utilizes the cooperative plan of educa- 
tion The students aie divided into two groups one in 
class at the college, the other in full-time work off cam- 
pus At the end of each three-month period they alter- 
nate The cooperative work experience received by the 
students is an important factor m the guidance program 
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Ob]ectin)es of the Guidance Program 

At the time the guidance progiam was first consid- 
ered m 1931 as an organized activity of the college, u 
seemed logical that the following objectives be kept in 
mind 

1 That the guidance piogiam he an integral pait 

of the college piogiam. 

The guidance progiam should perform so essen- 
tial a function that the contiibution and progiess 
of the guidance activities could best be judged by 
the progress made by the institution as a whole 

2 That the guidance pfogiam be cenieted on the 

noimal student 

In too many cases, because of lack of time and 
personnel, only the maladjusted individuals m 
college get any real assistance from the guidance 
program and the normal student is, to a large 
extent, disregarded By stiessing the preventive 
as well as the adjustment phases of guidance 
woik, and by developing techniques and methods 
which would be helpful to the noimal students, 
it was hoped that the piimaiy function of the 
guidance program would be to help the normal 
individual 

3, That all faculty membeis participate 

It was thought desirable to have eveiy faculty 
member share in the piogram so that all students 
could be assisted properly It was assumed that 
every faculty member could do some formal 
guidance work and that in doing his part he 
could, if given propel help, progiessively qualify 
himself to do moie and to do it better. Further- 
more, It was assumed that the counseling expe- 
riences could help him to become a more effect- 
ive teacher Therefore, participation was ex- 
pected to be of personal value to the instructor 
who shared m the program 
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Oi ganization of the Guidance Program 

During the past ten yeais, a guidance progiam was 
evolved which is in line with the above objectives A list 
of the features of the progiam, with a brief description, 
follows 

1 The guidance ptogtam staits with the student 

It helps him face as much of the responsibility 
foi directing, motivating, and appraising the per- 
sonal development as is educationally desirable 
It expects and requires progressively greater as- 
sumptions of that responsibility as the student 
becomes more expeiienced and more capable 

2 Each instraciO'i assumes a shaie of the respond 

sibihty in the guidance program as a general 

cQUnseloi 

Each instiuctor acts as a general counselor for 
at least 10 students The counselor is the insti- 
tution’s representative who assumes the respon- 
sibility of seeing that eveiythmg within the powei 
of the college is done to help the student carry 
out his piogram of development to a successful 
conclusion All foimal guidance woik, except in 
abnormal oi unusual cases, is cained out through 
the counseloi 

3 A guidance specialist is piovided to seive as a 

supeivtsof of the counseloi s 
His lesponsibility is to oigamze the program, to 
select and develop techniques, and to provide 
leadership and diiection to the guidance pro- 
gram, 

4 Each fieshman student is em oiled in a group 

guidance classj called the Orientation Glass 
This feature will be described in detail later in 
this aiticle 
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5 Faculty case boaid confei ences aie held at iht 

end of each qua} lei 

At these confei ences the woik of all students is 
leviewed and the progiess and difficulties dis- 
cussed This activity is a very effective part of 
the guidance progiam and piovides very good in- 
set vice Uaining media m guidance techniques 
and methods foi counselors 

6 Vat tons useful data aie collected and made avail- 

able to both the student and the counselor 
These data include test results, records, and other 
vital information 

7 A clinic IS piovided to deal with piohlem 

students. 

Specialists pai ticipate in this clinic This feature 
IS still in its eaily stages of development 

The Recoid and Planning Folder 
Space does not allow a detailed description of each 
featuie of the piogiam However, the Record and Plan- 
ning Folder and the Orientation Class, because of their 
uniqueness and impoitancc in the guidance piogram, will 
be discussed heie in greatei detail 

The common practice is for colleges to keep records 
of the student’s plans, achievements, and experiences This 
practice takes care of the administrative needs, but does 
not give the student an opportunity to learn how to keep 
his own lecords Planning lequires that reliable mfoi- 
mation be gathered, organized, and used For that rea- 
son It lb important that the student leain how to keep 
recoids as a part of his training in self-guidance 

Thiec years ago a group of students in an orientation 
class decided to do some pionceimg work in the aiea of 
personal record keeping. The foim developed was called 
the Recoid and Planning Folder. The students found 
the lecord veiy helpful and weie quite enthusiastic about 
its value. The folder provided a convenient method of 
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gathering and organizing information necessary for effect- 
ive self -guidance 

Most of the infoimation used in the folder was already 
available but seldom organized by the students The fol- 
lowing items weie placed in the Record and Planning 
Folder on specially piepaied forms during the freshman 
year' 

1, Peisonal htstoiy 

This section includes personal data, such as date 
of birth, fathei’s and mother’s names, occupa- 
tions, nationality, names and ages of brothers and 
sisters m the family, and the student’s employ- 
ment expenence prior to entrance in college 

2 Autobiography 

The autobiogiaphy is a report of approximately 
1,500 words containing the highlights of the stu- 
dent’s history 

3 Sum maty of high school lecoid and experiences 
This summary includes all the subjects taken by 
the student and the grades, listed chronologically, 
as well as the student’s rank m class, honors and 
scholarship, special courses, extracurnculai ac- 
tivities, and his appraisal of his high school ex- 
periences 

4 Entrance test results 

Each freshman undergoes a two-day testing pro- 
gram as part of the freshman week activities The 
tests are of the type which the average instructor 
and student can undeistand and use For that 
reason, the information is made available to both 
the student and the counselor. Areas of testing 
and the tests given aie as follows; 

Geneial Ability Tests, ACE Psycholagicnl Exfiminatian and 

Otis Mental Ability Test 

General Background Tests Cooperative General Acliievement Tests in 

Natural Science, Social Science, Mathe- 
matics, and English 
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Training in Special Subjects Iowa Placement Examinations in Mathe 

matics and Chemistry Training 

Gencial Aptitudes Iowa Placement ExaminJitions m Matlic- 

maticb, English, and Cliemistry Aptitudes 

Vocational Inteiests, Strong’s Vocational Interest Test, using the 

modified scoring 

Special Skills Reading Test 

Peisonal Chaiacteiistics A batteiy of tests developed by the author 
5 Report on the feniative plans foi the school yea) 
This lepoit covers the geneial plans foi petsonal 
development that have been worked out with the 
the student’s counseloi In addition to the aca- 
demic plans, they include plans foL walk experi- 
ence, extiacuriiculai activities, social develop- 
ment, and community and leligious activities, 

0 Scholastic tecotd fot the fieshman yeai 

Subjects taken and the grades received duimg 
each quarter, point aveiages and rank m class, 
and the student’s appiaisal of the work done each 
quarter are included 

7 Goope) alive wotJi eocpeiience 

Freshmen noimally start on their coopeiative 
work expeuence at the end of the thud quarter 
Each student is lequued to write a report about 
this work expeuence The highlights of this 
repoit, expeuence received, earnings, the employ- 
ei’s evaluation of the student’s work, and the stu- 
dent’s appiaisal of the work experience, are 
placed m this section 

8 Record of unusual expei tences and opporiumiies. 

utilized 

This recoid includes extiacunicular and com- 
munity activities HI which the student engaged, 
worthwhile social and religious experiences, as 
well as any unusual activities 
Q Repoit on life philosophy 

As a part of the Orientation Class activities 
the student states his philosophy m terms of a 

284 



GUIDING STUDENTS TO BECOME SELE-GUIDING 


pattern of beliefs He inserts this in the Record 
and Planning Folder 

10 Appfaisal and emhiation of piogiess and expeit- 
ence during school year 

In this section the student recoids the highlights 
of the appiaisal he and his counselor have made of 
bis pj ogress and difficulties, and modifications 
of plans made as the year progressed 

Provisions in the Record and Planning Folder are 
made for the following information for each succeeding 
year 

1 Addition to the autobiography 

2 A leport on tentative plans for each year (Made 
prior to registiation ) 

3 Scholastic record 

4 Coopeiative woik (work experience record) 

^ Record of unusual experiences and opportunities 
utilized 

6 Appraisal and evaluation of progress during the 
school year (Made at the end of each school 
year } 

The Record and Planning Folder might very easily 
become one of the most significant features of the guid’ 
ance piogiam because it is helpful in so many different 
ways to the student himself and to the faculty members 
who deal with him This coming school year every stu- 
dent will maintain the Record and Planning Folder as a 
part of his own guidance efforts 

The Orientation Glass 

The purpose of the Orientation Class is to help stu- 
dents intelligently plan and carry out a program of per- 
sonal development that will lead to successful adjustment 
in major areas of adult responsibilities It is an activity 
which provides opportunity for formal training in the 
process of self-guidance 
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The students and mstriictots jointly and mutually 
assume lesponsibihty for planning, organizing, conduct^ 
jng, and evaluating the Orientation Class activities Con- 
sideiation of pioblems of personal adjustment and per- 
sonal development makes up the piogiam Three weeks 
are spent m planning the piogiam, seven weeks m carry- 
ing It out, and one week m evaluating it 

The general theme of the course is "Leaining to Live 
m Oui Modern Complex Society^’ The aieas of adult 
activities which have been chosen to constitute the points 
of emphasis of the course are as follows Learning to Hve 
with (1) one’s self, (2) others, (3) one’s job, (4) one’s 
government, (5) one’s estate, (6) one’s ciiltuie, and (7) 
one’s family 

Each student as a member of a committee helps plan 
a piogram consisting of three one-houi sessions m one 
of the above seven areas that he chooses, and then helps 
conduct the class activities At the end of the week, each 
student turns m a lepoit which includes his objectives, 
his problems, and his plans for peisonal development in 
the particular area undei discussion By the end of the 
quarter each student has wiitten in detail a leport con- 
taining seven sections stating how he intends to utilize 
the opportunities for personal growth to be found m col- 
lege, vn his cooperative woik, and in community activities 

The advantages and importance of such a group guid- 
ance activity are readily seen In the first place, the 
students are oriented into the major areas of adult life 
activities and responsibilities, in the second place, they 
are given a demonstration, through group thinking, of 
how a student, by means of choice and planning of activi- 
ties, learns to assume more icsponsibility, exeicises a 
greater use of his intelligence, attains greatei control of 
his behavior, and is able to evaluate his experiences more 
meaningfully, than the student who merely drifts with 
the current of events in college. Finally, in a friendly, 
informal atmosphere, a stimulating environment is pro- 
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vided for the student so that he gets a good start on his 
self-guidance program 

Difficulties Encountef ed 

Difficulties that are encountered in the guidance pro- 
gram are common and familiar to all guidance workers 

One of these difficulties is that of making the term 
‘'leaining self-guidance” more concrete, both as to what 
IS to be learned and how it is to be practiced We have 
found that a practical approach to the difficulty is to limit 
the formal training the students are to receive in the area 
of self-guidance to the following three aspects* 

1 The development of a dynamic outlook on life — 
as a source of direction and motivation, 

2 The acquisition of a basic knowledge of the plan- 
ning process — as a means of organizing and direct- 
ing one’s efforts 

3 The maintenance of a record — as a means of evalu- 
ating one’s piogress and as a means of interpreting 
one’s effoits to others 

Each year that the student is m college he has an 
opportunity to formulate and discuss with his counselor 
the objectives of personal development he wishes to 
achieve during the year and plans for achieving those 
objectives, as well as any modifications of his plans or 
objectives made during the year At the end of the year, 
he and his counselor evaluate the progress made If the 
student follows this procedure each year he is in college, 
he will have practiced self-guidance m a very effective 
and worthwhile way 

Another difficulty faced is that of getting the instruc- 
tors, who are busy and often not too interested or qualified 
in guidance work, to put the necessary effort into the job 
We have tried to minimize the “too busy” problem by 
(1) making the student more active m the program, (2) 
making the information about students quickly and easily 
available and usable, (3) distributing the task equally 
among all the instructors 
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The ''not iiiteiested” problem was tackled by (1) ex 
pectmg and giving the instuictois an opportunity to par- 
ticipate on the thcoiy that paiticipation stimulates inter- 
est; andj (2) getting them to see that the guidance pro- 
gram contiibutes to the iinpiovement of then mam func- 
tion, which IS teaching. 

Finally, the "not qualified” pioblem was handled by 

(1) providing in-seivice training for the instuictors, and, 

(2) simplifying the techniques and devices to a point 
where the average teachei can use them 

Anothei difficulty is that of overcoming student indif- 
ference and the tendency to drift Guidance implies mo- 
tion. Self-guidance, theiefoie, implies self-piopelled mo- 
tion It is absolutely essential in the guidance piogram 
that the student take the initiative in developing himself. 
The Oiientation Class, the counseling system, and the 
Recording and Planning Foldei help motivate the stu- 
dent to take the initiative, lathei than to sit back and 
diift with the cm lent of events 

It IS not possible to make an evaluation at this time 
of the complete effectiveness of the guidance piogiam 
A survey of the results up to this date would show that 
about 20 pel cent of the students and 40 per cent of the 
faculty niembeis aie doing a good job ol their lespective 
parts of the piogiam. Almost one- thud of the faculty 
and one-foLiith of the student body aie not functioning 
very effectively The remainder are doing just a fair 
job Admittedly, progress has been slow But the par- 
ticipants are becoming more and moie interested as time 
goes on, and the program appears to be growing in ef- 
fectiveness. Progress should be more rapid within the 
next five years now that all the featiues desciibed are 
included m the program 
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THINKING 


MAXD ENGELHART AND HUGH B LEWIS 

Cliicigo City Junior Colleges 

P OSSIBLY the most challenging pioblem facing those 
engaged in the construction and use of objective tests 
IS the cieation of exeicises which will require the func- 
tioning of abilities transcending memory The senes of 
exercises piesented m this paper may notdeseive the label 
of a test of the ability to think scientifically It seems 
justified, however, to piesent the senes in the hope that 
the form of the exercises may suggest to other and moie 
ingenious test makers improved means of measuring abil- 
ities which aie among the uinveisaily recognized goals 
of science instruction 

The senes of exercises given heie follows m its organ- 
ization the steps often regarded as the essence of the 
scientific method One would be naive to believe that 
scientific pioblems aie always solved m just these steps, 
01 that the piocesses involved in their solution may not be 
more complex On the other hand, the use of the stages 
lepiesented in these exercises may be appropnate in test- 
ing students Although the pioper function of a test is 
measurement, it may still be legitimate to ‘recognize the 
function of motivation, and exercises of the type classified 
may also accomplish the piupose of engendeiing in the 
minds of students a belief that knowledge of the scientific 
method is important 

When the exercises were constructed, it was felt essen- 
tial to present ceitain introductory statements descriptive 
of the scientific method and of the phenomenon with 
which the problem to be solved is concerned The distmc- 
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tion between the directness and indirectness of the contri- 
bution of a datum in detei mining the tiuth or falsity of 
an hypothesis may be somewhat aitificial It is possible 
that the use of three categoiies lathei than five would be 
justifiable. One might aigue, however, that scientific 
thinking does involve this evaluation of lelevaiicy of data, 
and that the moie duect the contubution, the greater is 
the dependence which may be placed upon the data 

When the content of the exercises was selected, an 
effoit was made to present a phenomenon which m most 
respects would be novel to the students — that is, the phe- 
nomenon would be novel, but the concepts basically in- 
volved would 1 elate to subject mattei or principles with 
which the students had had some expeiience The exei- 
cises on the opeiation of the radiometer were developed 
from a senes of exercises of a somewhat different type 
which were wiitten by Dr C E Ronneberg of Herzl 
Junior College foi the January, 1939, physical science 
compiehensive examination, It is possible to select other 
phenomena for which pioblems can be stated, and to con- 
struct similar exercises There is, of course, no necessity 
to restrict such exeicises to the field of physics The par- 
ticulai phenomenon and exercises presented here were not 
an altogether appiopriate selection so far as the group 
tested was conceined, since the level of difficulty was 
too great 

The senes of exeicises was included in a test adminis- 
tered to students entering the Chicago City Junior Col- 
leges who wished to enroll m the second, rather than in 
the first, semester of the physical science survey The 
exercises and their introductory mateiials are reproduced 
below 

A scientist, wlieii confiontcd with a pioblcm, formulates hypotheses 
which represent tentative solutions to the problem He then collects 
data which may support oi disprove his hypotheses Finally, on the basis 
of the data and the hypotheses thus tested, he derives a conclusion which 
constitutes his answer to the problem 
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The following exercises represent an effort to test your ability to do 
scientific thinking You are to test certain true or false hypotheses, and 
to evaluate ceitam gcneial conclusions Assume that each item of data 
below eacli hypothesis is a true statement and may directly or indirectly 
help to piove an hypothesis tiue or false 

If the application of the item of data leijuircs only one step to prove 
the truth oi falsity of an hypothesis, then the item is a direct help For 
example, the tempeiatuies of water boiling on a given mountain and 
at sea level would lepresent dnect evidence of the falsity of the hypothe- 
sis "watei boils at a highei tempeiature on a mountain than at sea level ” 

If the application of the item of data lequires moie than one step to 
prove tile tiutli oi falsity of an liypothesis, then the item is an mdirecl 
help For example, the item ‘‘water in a container tliat can be evacu- 
ated will boil at loom temperature'* indijectly helps to prove the falsity 
of the hypothesis "water boils at a higliei temperatuie on a mountain 
than at sea level ” 

A number of years ago Sir William 
Ciookes perfected an instrument 
which always intiigues people, 
whethei laymen or scientists This 
is the radiometei, a device consist- 
ing essentially of a paddle wheel 
which IS fiec to rotate in a hori- 
zontal plane within a partially 
evacuated glass bulb One side of 
each paddle is brightly polished, 
while the other side is coated with lampblack As soon as the device 
IS placed in the sunlight, the little paddle wheel starts to spin rapidly 
It continues to spin until the device is again placed in the dark 

PROBLEbd" How does sunlight cause the paddle wheel to rotate? 

Below are given a senes of hypotheses, each of which is followed by 
numbered items which icprescnt data After each item number on the 
answer sheet blacken space 

A if the item diiectly helps to prove the hypothesis tiue, 

B if the item indirectly helps to piove the hypothesis true 
C if the Item directly helps to prove the hypothesis false 
D if the Item indirectly helps to piove the hypothesis false, 

E if the Item neither directly nor indirectly helps to prove the hy- 
pothesis true or false 


A P«llihBd 
face 



“A 

foea 


PADDLE WHEEL FROM ABOVE 
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HYPOTHESIS I In a partial vacuum the paddle wheel rotates be- 
cause of the impact of photons of light 

128 Scientists now believe that light has both corpuscular and wm 
cliaiactcnstics 

129 In A vciy high vacuum the hiight faces of the paddle wheel turn 
slowly away fiom the light, while the black faces turn toward the 
light 

130 Light tiavels at the late of 186,000 miles per second 

131 In a paitial vacuum the black faces of the paddle wheel turn away 
from the light, while the bright faces turn toward the light 

132 Light tiavcls at a slowei speed in glass than in air or in a vacuum 

133 After this item number on the answer sheet blacken space A if 
Hypothesib I is true, oi space B if it is false 

HYPOTHESIS 11 A paddle wheel on winch all of tlie faces are 
blight or till aic black will not rotate 

134 The black faces of paddles absorb energy from light to a greater 
extent than the bright faces of paddles, 

135. Rotation is due to force of riiipact If all paddles are the same on 
both sides, cither all blight or all black, the turning forces would 
cancel 

136, More pliotons rebound from blight faces than from dark faces 

137 In a paitial vacuum, an molecules are constantly hitting the 
paddles 

138 Photons are hitting the sides of the paddles winch face the light 

139 After this item number on the answer slieet blacken space A if 
Hypothesis II is true, or space B if it is false 

HYPOTHESIS III * Rotation nr a partial vacuum of the paddle wheel 
IS due to the gi cater force of rebound of air molecules from the 
black faces tlran from the bright ones 

140 The bright faces remain cooler than the dark faces, since they 
reflect more light 

141 In a partial vacuum and in the daik the paddle wheel will rotate 
when exposed to invisible infrared rays from a warm flatiron 


292 



AN ATTEMPT TO MEASURE SCIENTIFIC THINKING 


142 The black faces of the paddles become warmer than the bright 
faces, since they absorb more light 

143 Air molecules adjacent to the warmer black faces lebound from 
these faces with greatei energy than from the cooler bright faces 

144 In a very high vacuum and in the dark the paddle wheel will rotate 
slowly if invisible ia3fs from a cathode tube aie directed toward it 

145 After this item number on the answer sheet blacken space A if 
Hypothesis III is true, or space B if it is false 

Below arc five conclusions After each coi responding number on the 
answer sheet blacken space 

A if in your judgment the conclusion is the best answer to the 
problem 

B if in your judgment the conclusion is neither the best answer noi 
the least satisfactory answer to the problem (Thice conclu- 
sions should leceive this mark,) 

G if in your judgment the conclusion is the least satisfactoiy 
answer to the problem 

146 The paddle wheel of the radiometer rotates, because air molecules 
move with gieater eneigy when heated by energy from sunlight oi 
fiom infrared ravs from a flatiron 

147 Air molecules rebound with gieatei foice from the biight faces, 
which reflect more light energy Photons rebound from dark 
faces to a gieater extent than from bright faces The turning 
forces thus cieated cause black faces to rotate toward the light 
in a partial vacuum and away fiom the light in a very high 
vacuum 

148 The paddle wheel of the radiometer rotates, because photons of 
light strike air molecules with greater energy when adjacent to 
the dark faces than when adjacent to the bright faces 

149 The fact that a ladiomctei will operate in either a paitial oi a 
very high vacuum demonstrates that it is not essential that air 
molecules be present m ordei to cause rotation 

150 Air molecules lebound with greater foice from the black faces, 
which absorb moie light energy than the bright faces Photons 
rebound from bright faces to a greater extent than from daik faces, 
The turning forces thus created cause black faces to rotate away 
from the light m a partial vacuum and toward the light in a very 
high vacuum 
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In the table below aie ptesented the proportions of 
correct response and coirelations of each of the items with 
the total score on the test and with the total score on the 
partj 1 e j the total score on the 23 scientific thinking exer- 
cises These data aie based on an analysis of the answer 
sheets of a random sample of 200 cases The selection 


TA.t|LE 1 

ANALYSIS OF ITEMS 


Item No 

Key 

Item Difficiiliy 

Item-Teat Cotrelation 
Correlation withCori elation t^ith 
Total Score on Total Score 
Entrance Test on Part 

128 

E 

17 

25 

35 

129 

D 

09 

IS 

20 

130 

E 

55 

53 

61 

131 

C 

18 

15 

27 

132 

E 

42 

37 

55 

133 

B 

21 

18 

30 

134 

E 

12 

19 

32 

135 

A 

60 

41 

41 

136 

E 

18 

15 

33 

137 

E 

28 

40 

S5 

138 

D 

23 

28 

40 

139 

A 

65 

40 

38 

140 

B 

22 

00 

27 

1+1 

£ 

18 

27 

27 

142 

B 

20 

08 

36 

143 

A 

41 

28 

33 

144 

E 

24 

41 

48 

145 

A 

33 

09 

29 

146 

D 

41 

38 

.38 

147 

G 

15 

20 

20 

148 

B 

50 

35 

45 

149 

B 

39 

09 

19 

ISO 

A 

39 

19 

17 


was made fiom the answer sheets of all of the students 
taking the test on entrance into the junioi colleges The 
reliability of the series was found to be 72 by means of 
a Kuder-Richardson formula The series of exercises 
correlated .64 with the total score on the entrance test 
Seventy-five of the othei exeicises were factual, multiple- 
answer items dealing with high-school physics and chem- 
istry, and 57 were true-false items pertairuag to sever al 
passages selected from advanced texts m the physical 
science field These latter exercises emphasized aptitude 
more than training m that they were essentially a reading 
test in the field of physical science. 
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AN EVALUATION OF TECHNIQUES OF MEASUR- 
ING VISUAL ACUITY AT THE COLLEGE LEVEL 

FRANCES ORAlIND TKIGGS 
University of Minnesotn 

KAHL E SANDT, M D 
University of Minnesota Health Service 

T he university of Minnesota Health Service and 
the Univeisity Testing Bureau have been coopeiating on 
an evaluation of the Betts Ophthalmic Telebinoculai Test to 
determine whether it is a valid screening test of visual acuity 
for use with college students 

The problem of determining what students should and 
what students should not be referred to an eye specialist is a 
real one because many health sei vices do not have such doctois 
on their staffs and students, many of whom cannot afford it, 
must pay for such service individually If there is an eye spe- 
cialist on the staff of the health service, it is difficult foi him 
to give each student individual attention The students who 
come voluntarily to him to be examined may be the very ones 
who do not need an examination, and those who do need it 
may never get it, for often a student himself does not know 
when his eyes need attention 

The plan of the research was this The Betts Telebinoc- 
ular Examination was included as a pait of the diagnostic 
reading test battery which is given by the University Testing 
Bureau m cases of suspected leading difficulties At the time 
the student took the Tclebinocular Test, he was given a note 
addressed to Dr K E Sandt at the Health Seivice asking for 
a complete eye examination This note was an indication to Dr 
Sandt that the student was to be included in the research It 
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was intended to have complete data from these tests on 100 
students, but when the data from the records on the Telebinoc- 
ular and from the Health Service lecoids were tabulated, it 
was found that data were complete for only 87 students The 
measiTies on which data were available from both sources 
weie visual acuity in the light and left eye separately, 
exophona and esophona neai and far, and hyperphoria near 
and fai If a student wore glasses he was supposed to be 
tested with and without glasses If the student wore glasses to 
the examination at the Health Service and the examination at 
the University Testing Buieau, the data with glasses were 
always used If data weie available only from one examination 
without glasses, the data fiom both scivices without glasses 
were used ^ 

In evaluating these data, certain facts should be kept in 
mind The Betts TelebinociilaP purports to screen out students 
having measurable visual defects serious enough to be con- 
sidered for correction by an opthalmologist In a letter to the 
University Testing Bureau fiom the Bureau of Research of 
the Keystone View Company dated Apiil 22, 1940, the follow- 
ing bases for icfenalwcic given “(a) You need have no hesi- 
tation about referring a patient who fails on any part of Test 
3, provided that if theie has been a failure on B or C with 
both eyes open you make the test by occluding the eye not 
being tested If there is still a failure the patient should be 
referied (b) Test 4 is seldom failed but if it is failed, with- 
out question the patient needs attention (c) The failure 
of Test 5 alone is not a waiiant foi referring the patient, 
but if there have been other failiues, particularly in Test 2 
and 6A, there is no question but what there is poor eye co- 

should be remembered when interpreting these data that there la no 
indication as to whether or not this group of students is a selected sample of the 
whole student body as far as visual acuity is concerned There is no reason to 
believe that they would be a selected sample on the criteria of visual acuity 
jusL because they are on rending skills, for it has been shown that there is no 
consistent relationship between these two factors 

^Por complete description of the instrument, see Emmett Albert Betts, The 
Prevention and CarrecUon of Readmff DifficuUies (Evanston, Illinois Row, 
Peterson and Company, 1936), pp 327-50 
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ordination (d) With high school and university students 
who complmn of discomfort in leading, failiiies of Test 6B 
and 7 taken together are indicative of near point trouble, 
and they should have attention ” It is upon these bases that 
referral was detei mined by the Univeisity Testing Bureau for 
this study The ophthalmologist is tiained to determine visual 
defects and decide whether they are seiious enough for correc- 
tion, thus it may be seen that theie is some overlap of service 
but no oveilap of lesponsibility, the final decision always lying 
with the ophthalmologist as to whether correction shall be 
given In the light of these stated purposes of the Betts test, 
it would seem that the following questions might be helpfully 
answered 

1 Will the Betts test screen out for referral to the oculist 
a large number of students in whom the oculist will find 
deficiencies serious enough for correction? 

2 Will the Betts Telebinocnlar refer a large proportion 
of students whom the oculist finds to have no measurable eye 
difficulty? 

2 By comparing the oculist’s and Betts’ records, on 
individual tests, aie all of the Betts measures equally satis- 
factory ? 

4 On what tests of the Betts Telebinoculai arc referrals 
made most often? Can a better basis of referral on the Betts 
Telebinocular be found than the one furnished us by the Key- 
stone View Company? 

The question which always arises in research of this kind 
is whether the tests measuie what they purpoit to measure and 
whether, if they were administeied a second time, they would 
give the same results as they did the first time These ques- 
tions have never been finally answered for either of the tests 
used in this study It may be that they never can be answered, 
for in both cases the results are dependent upon the physiolog- 
ical status of the individual being tested The relationship 
of the effects of fatigue, light, and other factors upon different 
individuals may vary so greatly that a constant score may not 
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be possible Or Lt may even be that there aie still to be dis- 
covered bettei ways of diagnosing visual anomalies 

In this study, the extent to which the two tests agiee on 
diagnosis is indicated, but neithei test is assumed to give a 
perfect diagnosis However, because piesciiptions aie finally 
made by the oculist, the extent to which the Betts would 
tefei to the oculist those people found by him to need correc- 
tion IS pointed out 

The following evaluation of data Is presented in answer 
to these foul questions 

1 Of the 87 students included m this study, 13 weie given 
glasses by the oculist, 11 weie given piescriptions to correct 
measuiablc physical eye defects, and two students weie given 
glasses meiely to impiove comfoit while leading rather than 
to coriect measuiable eye defects 

Of the 11 students given piesciiptions to correct measur- 
able physical eye defects, all would have been lefcrred on 
the cnteiia of refeiral sent us by the Keystone View Company 
The lemaining two students would not have been lefened on 
the basis of these ciiteiia Thus it will be seen that all students 
found by the oculist to have measuiable physical eye defects 
would have been rcfeired for complete examinations as a 
result of the Telebmoculai Test 

2 Of the 87 cases which had both the Betts test and an 
ophthalmic examination, 46 would have been lefeued by the 
Betts test to the oculist on the basis of the criteua furnished 
us by the Keystone View Company Of the 46 students who 
would have been lefeiied by the Telebinocular Test, only 11) 
01 24 per cent, had defects seiious enough to be corrected 
by glasses Howevei, it should be rcmembeied that while 
only about S3 pei cent of the gioup would have been refcired 
to the oculist foi complete testing, 100 pei cent would have 
had to be tested, had no pie-test been given While it might 
be desirable to have a moie rigorous screening test, it is cer- 
tainly worth-while to save the oculist from having to examine 
almost half of the students 
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3 Thei data which be.ii on this question are presented 
in Table 1 Data weie complete from both sources for 87 
students on visual acuity for the right and left eyes For the 
light eye, we find that 20 students, or 23 per cent, failed the 
Betts test and the oculist’s test, 48, oi 55 per cent, passed 
both tests In othei woids, it was found that the oculist’s 
diagnosis and the Betts' diagnosis agreed on 78 per cent of 
the cases Six students, oi 7 pei cent, failed the Betts test but 
weie found satisfactoiy by the oculist, and 13, oi 15 per cent, 
passed the Betts test but failed the oculist’s test It should be 
remembered that of these 13 who passed the Betts test but 
failed the oculist’s test, the defect found by the oculist was 
in no case consideied serious enough for coirection 


TABLE 1 

COMPARISON OI RDSULIS DETTS PESTS AND OCULIST'S TESTS FOR EIGHTY-SEVEN 

SUDJECTS 


Failed Betts 

and Ocufist's 

Passed Betts and Passed Oculist's 

Right Eye 

Left Eye 

Right Eye 

Left Eye 

No % 

No % 

No % 

No % 

20 23 

13 IS 

48 55 

54 62 

Fnileii Betts and Passed Oculist's 

Passed Betts and 

Failed Oculist's 

Right Eye 

Left Eye 

Right Sye 

Left Eye 


No No % No % No % 

6 7 10 115 13 15 10 115 


For the left eye, data were again complete for 87 cases 
We find that 13 students, or 15 per cent, failed the Betts 
test and the oculist’s test, 54, or 62 per cent, passed both 
tests Thus the oculist’s and the Betts’ diagnosis agreed on 
77 per cent of the cases Ten students, oi 12 per cent, failed 
the Betts test but were found satisfactory by the oculist, and 
10, or 11 per cent, passed the Betts test but failed the oculist’s 
test It should be remembered that of those 10 who passed 
the Betts but failed the oculist’s tests, the defect found by 
the oculist was in no case considered seiious enough for 
correction 

On the measure of vertical imbalance (hyperphoria) far 
point on the Betts test, none of the 72 cases on which data 
are complete for both measures failed the test (Two stu- 
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dents who weie included in the study, but for whom no oculist 
measure of vertical imbalance far point was available, failed 
this test ) The oculist found that 10 students, oi 14 per cent, 
of the 72 students foi whom data weie complete on both meas- 
uies had veitical imbalance and 62, oi 86 pei cent, did not 

The Betts test foi veitical imbalance neai point did not 
identify any member of the gioup with this difficulty, but the 
oculist found that seven, oi 10 pei cent, of the 72 did have 
veitical imbalance and 65, oi 90 pei cent, did not have vertical 
imbalance neai point These data would seem to indicate that 
this test IS of questionable value foi use with college students 

On the lateial imbalance near point, data are complete 
for 72 cases Of the 72 cases, foiii students, or six pei cent, 
failed the Betts and the oculist’s tests, two, oi thiee pei cent, 
failed the Betts and passed the oculist’s test, 18, oi 25 per 
cent, passed the Betts and failed the oculist's test, and 48, or 
66 pel cent, passed the oculist’s test and the Betts test Thus 
It will be seen that the two measuies agieed in 72 pei cent 
of the cases 

For lateral imbalance far point, no students of the 72 who 
weie found to be unsatisfactoiy by the oculist faded the Betts, 
but two students, oi three per cent, failed the Betts who were 
found to be satisfactoiy by the oculist; foui, oi five per cent, 
passed the Betts test who weie found to be unsatisfactory 
by the oculist, and 66, oi 92 pei cent, weie found to be satis- 
factory on both mcasuies Foi lateial imbalance far point, 
the two measures agieed in 92 per cent of the cases 

This evaluation of tests would Lead us to say that, of the 
parts of the Betts tests studied, the one found to be most 
valuable for lefenal of college students for a complete eye 
examination is the one of visual acuity for the light and left 
eye There is not complete agieement between this test and 
the oculist’s measiuements, but it does agiee 78 out of 87 
times for the ughl eye and 77 out of 87 times foi the left eye, 
and wheie it does not agiec the oculist found no situation to 
exist which was serious enough foi conection This finding 
raises the question as to whether anothei measure of visual 
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acuity not requiiing expensive apparatus would serve as satis^ 
factorily Only one othei measure of visual acuity was given 
these students When students enter the University of Minne- 
sota they aie requiied to have a physical examination at the 
Health Service As a part of that examination the eyes are 
checked by use of the Snellen Chart® The recoid of these 
e'caminations was on file at the Health Service and has been 
tabulated for consideiation here 

There was a recoid of a Snellen examination for all the 
87 Students included in this study On the basis of that exam- 
ination eight students would have been leferred to the oculist 
for complete examination For this study it is important to 
deteimine whether these aie the same students referred by the 
Betts test and also to asceitam in how many cases the students 
refer! ed were found by the oculist to have a defect serious 
enough to wan ant a prescription 

Of the eight students refeired by the Snellen test, six 
would also have been leferred by the Betts As has been 
stated, 13 of the 87 students weie found by the oculist to 
have defects seiious enough to be corrected by glasses Of the 
13 given glasses, the Snellen Chart examination would have 
referred only two to the oculist These data would seem to 
indicate that on measuies of visual acuity the Betts test is 
superior to the Snellen Chart m refeiiing students with actual 
difficulty to the oculist foi complete examination 

4 It will be lemembeied that the Keystone View Com- 
pany gave four standaids for refeiring a student to the oculist 
on the basis of the Betts test These were 

(a) Faiiuie on test 3 o; any pait of test 3 (visual acuity) 

(b) Failure on test 4 (veitical imbalance) 

(c) Failure on test 5 (cooidination) with failure on test 
2 (distance fusion) and failure on test 6A (lateral 
imbalance fai point) 

(d) A complaint of discomfort m reading with test 6B 
(lateial imbalance neai point) and test 7 (fusion at 
reading distance) 

Hb\d , pp 1+9-51 
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Of the 46 students who would have been referied to the 
oculist on these standaids of lefeiial foi the Betts test, 43 are 
identifiable by then faiLiue on the first cuteiioti, i e , failure on 
one or more of the visual acuity tests Two of these students 
also failed the veitical imbalance fai point test Two of the 
students who would have been refeiied failed on both criteria 
one and foiii Two faded cuteiion foiu only and would 
have been lefeiied on it alone One student failed and would 
have been lefeired on ciiteiion thiee alone 

Thus, examination of oiii data would indicate that referral 
on the basis of ciiteria one, three, and four and the recoid of a 
complaint of discomfituie would have leferied all students 
given prescriptions by the oculist to coirect measuiable phys- 
ical eye defects The numbei lefciied wovild have been 46 
and it would have included the same students icfeiicd by the 
four ciitena given by the Keystone View Company It is 
questionable whether the test of veitical imbalance adds any- 
thing essential to this senes when used as a screening device 
at the college level 

On the basis of the data just piesented It does seem that 
the Betts Ophthalmic Telebinoculai Test can be used satis- 
factorily by colleges as a scieening device foi leferral of cases 
to the oculist The Betts test also stands up more satisfactor- 
ily than does the Snellen Chait examination (when given 
under the conditions described) as a measure of visual acuity 
These conclusions should be checked by repeating this study 
on another gioup of students, and they should be accepted 
only tentatively for situations other than those described in 
this study 



THE CONCEPT OF SCATTER IN THE LIGHT OF 
MENTAL TEST THEORY' 

MAURICE LOER 
TJ S Civil Service Commission 

RALPH K MEISTER 
Mooseheart Laboratory for Child Research 

T he confusion and loose thinking among clinical 
psychologists conceinmg the basis and significance of 
scatter on scales of the Binet type suggest a re-examination 
of the concept of scatter in the light of the theory of psycho- 
logical measurement 

Theoretically, on mental age scales of the Binet type, items 
are arranged m the order of their difficulty, the easiest item 
first In clinical practice these items are administered to a 
child in the same sequence Groups of items, supposedly equal 
in difficulty, are allocated to each year level as representing 
the typical performance of individuals of the corresponding 
chronological age The child is given increasingly difficult 
Items until he reaches a point m the scale above which he fails 
Actually, no such point exists Instead, the child passes all 
tests at a certain level and continues with mixed successes and 
failures on to the next higher level until he fails all items pre- 
sented to him m a given level Such a spiead of successes and 
failures over a numbei of mental year levels is called scatter 
Test theory indicates five possible bases for such irregu- 
larity of performance First, scatter is a consequence of the 
lack of perfect correlation between test items resulting from 
the presence of error and from the low communality and 

iThe authors wish to thank Dr M W Richardson for his review of this 
article and Dr M L Reymert for hia interest and encouragement throughout 
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high specificity of the items The error, as Mosier (10) has 
shown, inci eases in the individual case with greater hetero- 
geneity of Items Thus, an individual who passes one item at 
a given level may not necessarily pass a second item, either 
because the two items do not mcasuie the same function or 
because of eiroi involved in testing Illustrative of this lack 
of perfect correlation between test items is the Cattell and 
Bristol study (4) in which a mean intercori elation of + 32 
was found foi seven Binet test items, Wnght (15) found the 
mean intei con elation on 31 items to be + 61 

Secondly, there is the fact that the Items aie incorrectly 
allocated in the ordei of difficulty This might be expected 
in view of the fact that Teiman and Merrill (12), for exam- 
ple, although using ciuves of piopoitions-passing for each 
Item m the pielimmaiy giouping of items, had as their goal 
in the final grouping an I Q distiibution with a mean of 
approximately 100 It is likely that such a procedure re- 
sulted in a giouping of items only loiighly ordei ed as to 
difficulty Theiefore, although the grouping of test items is 
appioxiinately in the oidei of their difficulty m the sense that 
an Item at age four is definitely less difficult than one at age 
10, neveitheless, in adjacent gioiips there aie probably a great 
many inversions in difficulty Thuistonc’s study (14) on the 
absolute scaling of Binet items shows that items at any paiticu- 
lar age level vary considerably in difficulty, a finding contiary 
to the assumption of relatively equal difficulty among items 
at any one age level He found, too, inversions iii placement 
according to difficulty. Thus an item which is easier than 
anothei may be placed at a highei age level In fact, alloca- 
tion at the different age levels on the basis of difficulty is 
improbable since the items distribute veiy unevenly m abso- 
lute difficulty over the age levels On the basis of Burt’s 
data, Thuistone (14) says, “The test questions tiie more 
numerous at certain ages than at others For example, there 
are 12 questions that scale at par between the ages five and 
SIX, whereas theie are only foui questions that scale at par 
between six and seven ” Any kind of arrangement that requires 
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the same number of tests at each age level js unhkely to result 
in equal giadations of difficulty since the tebt items used 
do not scale into any such grouping 

This fact of the incorrect allocation of test items at the 
various age levels is brought out by the findings of many 
other investigators Cyiil Burt (2) admits that no two 
editors agiee about the coirect order of mental age items 
and cites instances Barber (1), on data for the revised 
Form L, found that five items were significantly easiei and 
SIX items weie significantly more difficult than their respective 
age placements would indicate Likewise, Harriman (5) 
found (for the Revised Scale) that test items at year level 
XII seem to be more difficult than those at year level XIII, 
a fact which is confiimed by Carlton (3) Krugman (8) 
found that for New York school childien, 25 of the items 
were incorrectly allocated 

Thirdly, scattei may be due to the lack of discriminatory 
power of certain items A highly diagnostic item will dis- 
ci iminate shaiply between individuals with ability above that 
required to respond coirectly to the item and those individuals 
who lack such ability For example, two items may be equal 
in difficulty (50 pei cent pass at, say, age 11), but differ 
widely in diagnostic value The psychometric curve for one 
Item may extend over, say, years five to 14 The curve for 
anothei item that is more discriminatory will extend over a 
much smallei age range, such as eight to 12 This spread 
01 scatter is manifestly a result of low diagnostic value 
Thurstone (14) plotted cuives of piopoitions-passing for a 
random selection of items fiom the Burt-Binet and found a 
"noticeable vaiiation in the slopes of curves ” It is probable, 
therefoie, that some of the scatter found is due to these dif- 
feiences in the diagnostic value of the items which these 
examples illustrate 

A fourth souice of scatter may be found m the tact that 
there is an increase in variability with an increase in absolute 
mean test performance (13) In other words, individuals 
apparently become moie vaiiable as they grow older Since 
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individuals vary moie among themselves, they must vaiy as to 
the number and types of items failed oi passed This can be 
easily seen, since the extent to which Thurstone’s “primary 
mental abilities,” foi example, aie present varies between 
individuals and within individuals, and the factorial composi- 
tion of test items diffeis fiom item to item within the same 
age level Thus an individual who passes one item at a certain 
age level may not pass anothei because the latter lequires an 
ability which he does not have to the lequued degree Again, 
an individual's failuie on five items at a ceitain level is no 
sure indication that he will fail the sixth, since the sixth item 
may require an ability which he has to a marked degree This 
tendency of scattei to mciease with mean test performance or 
chronological age is checked in actual piactice between the 
ages 10 and 12 (Rcymeit and Meistei [11]) for within that 
age lange the ceiling of the test begins to limit the amount 
of scatter possible 

A fifth possible cause of scattei is the piesencc of sys- 
tematic errors m testing due to language handicaps, sensory 
defects, special tiammg, lack of coopeiation, and ambiguous 
scoring or instiiictions Unlike chance eirois which influence 
the results as often in one diiectioii as in another and there- 
fore can be assumed to cancel out one another, systematic 
errors have a consistent and cumulative effect that gives the 
results a constant bias Obviously if a test shows constant 
bias for a given individual, it is unsuited foi that individual, 
1 C, the individual diffeis sufficiently from the norm popula- 
tion to render the test inapplicable If such a test is given, 
the person with language difficulty will tend to fail highly 
verbal items, the uncooperative individual will answer only 
those questioni, which he can be motivated to try, the individual 
with a slight hearing loss may miss the cutical part of the 
question, etc All of these factois tend to lower the basal 
level which per se gives a laigei amount of scatter 

The uses to which measures of scattei have been put can 
now be critically examined m the light of the souices of scat- 
ter given above Perhaps the first point that should receive 
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critical attention is that of the methods of measuring scatter, 
for there are a great many such methods and the amount 
of agreement among them seems to be inversely related to 
their number Theoretically, if scatter represents the range 
of uncertainty of an individual’s ability, the range within 
which lies the limen of his ability or the point beyond which 
he will fail all the items — a point which m actual testing 
practice does not exist — then scatter should be measured on 
a scale of absolute difficulty, as the distance between the 
easiest item failed and the most difficult one passed Actually, 
as no such measure has evei been used, it is not surprising 
that, to quote Hams and Shakow (6), “reseaich up to now 
has failed to demonstrate clearly any valid clinical use for 
such measures These authois mention nine methods of 
measuring scatter and conclude that “at the present time it 
IS impossible to state which is the best method of measuring 
scatter ” 

NoWi in view of the uncertainty about measures of scatter, 
the uses to which they have been put become all the more 
questionable It is common practice to use measures of scattei 
as indicative of epilepsy, psychosis, feeblemindedness, emo- 
tional maladjustment, hypopituitarism, etc Harris and Sha- 
kow’s paper (6), in fact, deals with “the possibility of obtain- 
ing clinically significant information from numerical measures 
of scatter ” 

Studies which have indicated significant differences m the 
mean scatter foi certain groups do not justify diagnosis of 
a particular condition in the individual case And when one 
consideis that for five papeis that repoit such differences, 
theie are four that do not (6), even differentiation for groups 
appears questionable For indicating the type of condition in 
question, there certainly should be a more refined instrument 
than scatter on a test designed to measuie intelligence 

From a consideration of the various bases for scatter, 
it is obvious that their influence is certainly gi eater than that 
of any chance errors Therefore, in thei light of these con- 
siderations, the use of scatter as even a crude estimate of the 
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measurement erior of the test for an individual is not an 
acceptable procedure 

It IS also common piactice among clinical psychologists 
to analyse the particular successes and failuies of an individual 
to give a crude appiaisal of his piimaiy abilities as well as 
estimates of mental dcteuoration in these abilities Such 
inspectJonal analysis is a questionable piactice for the fol- 
lowing reasons First, the factoiial composition of an item 
cannot be prejudged accuiately Such judgments are frequently 
in complete disagi cement with factoi analysis results For 
instance, Wright (15) found that items involving lepeatmg 
digits backwards did not necessaiily involve memoiy ability 
but rather other factois And yet how many failiues on such 
Items have been analyzed in lepoits as poor memory ability? 
Secondly, an item may be solved thioiigh the use of diffeient 
abilities by different individuals and at different age levels 
Thirdly, items may show fairly high loadings on more than 
one factoi so that failure cannot be attributed to the lack 
of any one ability Fourthly, such clusteis of items have too 
low a leliability to have any real diagnostic value 

In summaiy, it is concluded that scatter is for the most 
part due to factors inhcicnt in test construction plus certain 
systematic errors In view of these facts, it appears that 
the possibility of ever securing clinically significant information 
from measures of scatter based on age scales m cuirent use 
IS slight indeed 
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Adkins, Dorothy C “The Relation of Piitnarv Mental 
Abilities to Piefeience Scales and to Vocational Choice" 
Psycho meirika, V (1940), 316 (Abstract of a paper 
read at the September, 1940, meeting of the American 
Psychological Association ) 

Benge, Eugene J "Wanted More Logic and Less Guess- 
work in Hning Salesmen ” Sales Management, XLVIII, 
No 3 (1941), 18-20 

Companies who have records for many employees, past 
and present, could profitably make an analysis of job require- 
ments, It IS suggested that a cii tenon of job efficiency be set 
up and employees classified accoidingly An outline, based on 
these data, for constiucting a rating scale in teims of factors 
which can be elicited at time of application is presented Mini- 
mum scores on the rating scale are to be established accord- 
ing to scores made by employees It is emphasized that scores 
on such a rating scale should not be consideied alone but only 
in connection with other souiccs of mformatibn in hiring 
employees D A Peterson 

Blackwell, AM "A Comparative Investigation Into the 
Factors Involved in Mathematical Ability of Boys and 
Girls " British Journal of Educational Psychology, X 
(1940), Pt I, 143-53, and Pt II, 212-22 

A group of 100 boys and a group of 100 girls, ages 
ranging from 13 to 15 years, were given 10 tests of "mathe- 
matical ability" including arithmetical reasoning, analogies, 
three "spatial tests," three "geometric tests," and a test of 
algebraic computation and reasoning The mtercorrelations 
for each sex were factored separately Interpretations were 
attempted on the basis of the centroid matrices and on the 

’Edited b} Profeasor Forrest A Kingsbury 

311 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


basis of an orthogonal rotation as designed to insuic a general 
factor Sex differences weie found “The results of the study 
seem to confnm the complex nature of mathematical abil 
ity . ” Harold Bechtoldt 

Coombs, Clyde H “A Factorial Study of Number Ability " 
Psychomefuka, Yl (1941), 161-89 

In order to investigate ceitaiu hypotheses conceining the 
nature of number ability, and, secondarily, the nature of per- 
ceptual speed, a battery of 34 tests was given to 223 Chicago 
high school seniors and the data were factoied by the centroid 
method Seven primary factors were identifiable upon rota- 
tion Several deductions are made lelative to the interpreta- 
tion of the factois and relative to the consistency of the data 
with the hypotheses which weie to be tested (Courtesy 
Psycho me inka ) 

Coombs, Clyde H. “A Ciiterion for the Number of Factors 
in a Table of Intercorrclations “ Psychomelfika, V 
(1940), 315 (Abstract of a papei read at the Sep- 
tember, 1940, meeting of the American Psychological 
Association ) 

Cureton, E E “Testing in College Personnel Service “ 
Journal of Constilltng Psychology, IV (1940), 221-24 

A suivey of the purposes of a college peisonnel service 
and of the extent to which available tests lend themselves to 
such purposes The need for an adequately standardized dif- 
ferential intelligence battery is emphasized To show progress 
and to give information concerning the pattern of abilities 
and attainments, test scores should be directly comparable 
Suggestions are made on coordinating test production W 
A Varvel 

Dwyer, P, S “The Solution of Simultaneous Equations " 
Psychometnka, VI (1941), 101-29 

This paper is an attempt to integrate the various methods 
which have been developed for the numerical solution of 
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Simultaneous linear equations It is demonstrated that many 
of the common methods, including the Doolittle method, are 
variations of the method of "single division ” The most use- 
ful vaiiatioii of this method, in case symmetiy is present, 
appeals to be the Abbieviated Doolittle method The method 
of multiplication and subtraction likewise can be abbreviated 
111 various ways of which the most satisfactoiy foim appears 
to be the new Compact method. These methods are then 
applied to such problems as the solution of lelated equations, 
the solution of gioups of equations, and the evaluation of the 
inverse of a matrix (Courtesy Psycho metrtka ) 

Dwyer, P S "The Evaluation of Determinants ’’ Psycho- 
met tika, VI (1941), 191-204 

The numeiical evaluation of determinants with a modern 
computing machine is discussed Various methods are pre- 
sented and their lelations to each othei aie indicated The 
methods piesented paiallel those developed m the previous 
papers on "The Solution of Simultaneous Equations " Espe- 
cially emphasized are the Abbreviated Doolittle and the Com- 
pact methods Additional topics include the evaluation of 
partially symmetiic determinants by means of symmetric 
methods and the evaluation of determinantal ratios (Cour- 
tesy Psycho me tttka ) 

Guilford, J P "The Difficulty of a Test and Its Factor 
Composition " Psyckomeiftka, Y1 (1941), 66-77 

A factor analysis of the 10 sub-tests of the Seashore test 
of pitch discrimination revealed that moie than one ability is 
involved One factor, which accounted for the greater share 
of the variances, had loadings that decreased systematically 
with increasing difficulty A second factor had strongest load- 
ings among the more difficult items, particularly those with 
frequency diffciences of two to five cycles per second A third 
had stiongest loadings at differences of five to 12 cycles per 
second No explanation for the three factois is apparent, but 
the hypothesis is accepted that they represent distinct abilities 
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In tests SO homogeneous as to content and foim, wheie a single 
common facto i might well have been expected, the appeal ance 
of additional common factois emphasizes the impoitance of 
considering the difficulty level of test items, both in the 
attempt to inteipiet new factors and in the practice of testing 
The same kind of item may measiue dilferent abilities accord- 
ingly as it is easy oi difficult foi the individuals to whom it is 
applied (Coiiitcsy Psychoweiiika ) 

Guilford, J P “A Note on the Discoveiy of a G Factor by 
Means of Thuistone's Centioicl Method of Analysis” 
PsychometJ ika, VI (1941), 205-8 

A fictitious factoi matiix including 16 tests and three fac- 
tois, one of which was a g factoi, was presciibed Fiom it two 
typical factoi pioblcms, including eirois of sampling, weie 
derived Students in tiainmg, without awaieness of the factoi 
patterns, ai lived at essentially collect solutions by the use of 
Thuistone’s centioicl method with lotation of axes Eirors in 
the calculated factoi matiix weie vciy close in size to the 
sampling eiiors in the coij elation coefficients It is concluded 
that a g factoi need not escape detection by Thuistone’s pio- 
cedures if the cuteiia of complete simple stiucture are not 
demanded (Coiutesy Pryc/io me Hi ) 

Horst, Paul, "A Non-giaphical Method for Tiansfoiming an 
Aibitiary hactoi Matiix into a Simple Stiuctuie Factor 
Matiix ” Psychomt^luka, VI (1941), 79-99 

The most commonly used method of factonng a matrix of 
intercoiielations is the centioid method developed by L L 
Thui stone It is, howevei, necessary to tianstoim the cen- 
tioid matiix of factoi loadings into a simple stiuctuie matrix 
m order to facilitate the inteipietation of the factoi loadings 
Cun cut methods foi effecting this tiansfoimation aie chiefly 
graphical and lequiie consideiable expeiience and peisonal 
judgment This papei presents a new method for transform- 
ing an arbjtraiy factor matux into a simple stiucture matrix 
by methods almost completely objective The theory under- 
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lying the method is developed and appioximation procedures 
are derived The method is applied to a matrix of factor load- 
ings pieviously analyzed by Thiu stone (Courtesy Psycho- 
metnka ) 

Hoyt, Cyiil “Test Reliability Estimated by Analysis of Vari- 
ance " Psychometnka, VI (1941), H3-60 

A formula foi estimating the leliabihty of a test, based on 
the analysis of vaiiance theory, is developed and illustrated 
The data needed foi the required computation are the number 
of collect responses to each item and the score foi each sub- 
ject The results obtained from this formula aie identical with 
those from one of the special cases of the Kudei -Richardson 
formulation The lelationships of the new pioceduie to othei 
approaches to the problem aie indicated (Courtesy Fsycho- 
meti ika ) 

Karlin, J E “The Isolation of Musical Abilities by Factorial 
Methods “ Psychometnka^ Y (1940), 316 (Abstract of 
a papei lead at the September, 1940, meeting of the Amer- 
ican Psychological Association ) 

Lazaisfeld, Paul F (Guest Editor ) “Radio Research and 
Applied Psychology “ Journal of Applied Psychology, 
XXIV, No 6 (1940), 661-853 

This entile number is devoted to 21 articles dealing with 
pioblems of radio research (including two on magazine advei- 
tismg leseaich techniques), not separately abstracted heie be- 
cause of space limitations They aie classihed into the follow- 
ing five groups of papers I “Commercial Effects of Radio” 
(F Stanton, E Smith &E Suchman, M Fleiss, M Erdelyi) , 
II “Educational and Other Effects of Radio” (S Reid, J R 
Miles, G Wiebe) , III “Program Research” (J N Peter- 
man, H Schweiin, C Daniel, H C Link & P G Corby) , 
IV “General Research Techniques” (E A Suchman & B 
McCaudless, M Rollins, H Gaudet & E C Wilson, D B 
Lucas, R Franzen, P F Lazaisfeld) , and V “Measurement 
Problems” (P F Lazarsfeld & W S Robinson, C Daniel, 
W S Robinson, R Franzen) F A Kingsbury, 
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Lentz, Theodol e F , and Whitmei , Edith F “Item Synotiymi- 
/.ation . A Method foi Detei mining the Total Meaning of 
Pencil-Papei Reactions," Psychowetnka, VI (1941), 
131-9 

Items have been studied heicLofoie for their value as 
elements of paiticiilai tests to the neglect of moie funda- 
mental lesearch into the multiple potentiality of items This 
aiticle pioposes a method of giouplng items into “synonymies" 
comprising all of the items which coiielate with a given key 
Item These synonymies can be used foj inteipietation of the 
total meaning of the key item (1) by inspection of the con- 
stituent Items and (2) by coiielational study of obtained 
single scoies of Individual pci sons The method is illiistiated 
by foiu items with mtei- and nitia-coi'ielalLonSi and chaiac- 
tenstics of an ideal backgiound leseivoii of items aie pointed 
out (Couitesy Psyckouictuka ) 

Mai till) D. R “Mental Tests in Clinical Practice " Austial- 
foumal of Psychology and PhUosophy, XVllI 
(1940), 144-53 

llie aiithoi discusses the pin pose of mental testing in child- 
guidance woik and desciibes the batteiy of tests foi gencial 
intelligence, special abilities and disabilities, school achieve- 
ment, peisonality tiaits, and emotional stability in use at his 
clinic. Refeiences aie made to the leceiit contioveisy between 
Cattell and Veinon ovei the value of the Binet test 
A ViUvd — 

McNemai, Quinn “On the Sampling Eiiois of Factoi Load- 
ings " Psychomei^tkayyi (1941), 141-52 

The results of thiee empuical studies on the sampling 
fluctuation of centioid factoi loadings aie lepoited The first 
study is based on data wbich happened to be available on 8 
vaiiables foi 700 cases and which weie factoi cd to thiee fac- 
tors for siibsamples The second study is based on fictitious 
data for 2500 cases winch piovlded sepaiate analyses on 25 
samples foi each of thiee situations five vaiiablcs, one factor, 
live variables, two factors, and six variables, thiee factors, 
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The thud study, based on leal data foi nine vauables and 
7000 cases, involves sepainte factorization foi 25 samples of 
200 cases The thiee studies agiee in showing that the sam- 
pling behavior of fiist centioid factor loadings is much like 
that of coi relation coefficients, wheieas the sampling fluctua- 
tions for loadings beyond the fiist aie distuibmgly large 
(Coiiitesy Psycho7}iehtka) 

McNemar, Quinn “Moie on the Iowa I Q Studies ” Joinnal 
of Psychology, X (1940), 237-40 

In a leply to the Wellman-Skeels-Skodak Ltvitw[Psychoh 
ogical Balletin, XXXVII (1940), 93-111] of his original 
critique of the Iowa studies on enviionmentally-dctermmed 
changes in I Q , the authoi does not find it necessaiy to modify 
mateiially his pievious cuticisms IF A Fmvel 

McNemai, Quinn “On the Numbei of Factors” Psycho- 
V (1940), 315 (Abstiact of a papei read at 
the Septembei, 1940, meeting of the American Psycho- 
logical Association ) 

Poitei, E K “Criteiia of a Good Examination” Public 
Health Nuf sing, XXXII (1940), 558-64 

Steps in the constiuctjon of a test aie outlined Piinciples 
foi the constiuction of tests and test items are presented 


Schaefer, Willis C “The Relation of Test Difficulty and Fac- 
toual Composition Determined from Individual and Group 
Forms of Piimaiy Mental Abilities Tests.” Psychomet- 
uka,Y ( 1940) , 3 16-17 (Abstract of a paper read at the 
Septembei, 1940, meeting of the American Psychological 
Association ) 

Van Steenbeig, N J “Analysis of Mental Giowth of School 
Children” Psychometrtka,Y (1940), 314 (Abstract of 
a paper read at the September, 1940, meeting of the 
American Psychological Association ) 
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Dr. John C Flanagan has been granted a year’s 
leave of absence from the Cooperative Test Service in 
order that he may accept a commission as a leserve officer 
in the Army Air Corps Dr Flanagan will direct de 
velopmental researches and make piactical applications 
with regal d to problems of selection of Air Corps 
personnel 


The authors of the Chicago Reading Tests, Drs Max 
D Engelhart and Thelma Gwmn Thurstone, are con- 
ducting an investigation of the comparability of the 
norms of these tests from foim to form and also their 
comparability tvith the norms of the Metiopolitan and 
Stanfoid Reading Tests Each senes of forms of the 
Chicago tests was standardized independently in succes- 
sive years by administration to pupils in a lepresentative 
sample of 30 Chicago elementary schools Approxi- 
mately 8,000 elementary pupils took each form when it 
was given for standardization In addition, two forms of 
the sixth-, seventh-, and eighth-grade test were adminis- 
tered in successive yeais to approximately 8,000 Chicago 
high-school pupils, The assumption which was made 
and IS now being tested was that norms based on large 

♦NoUa fot thia departmenl should be sent to Dr M W, Richardson, United 
States Civil Service Cominissionp Washington, D C 

318 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


samples of pupils diawn from the same schools should be 
comparable In the cuneiit study each of the three forms 
of each of the foui Chicago tests, and the appropriate 
Metiopolitan and Stanford tests, weie administered to 
several bundled pupils in landomized order to control 
practice effect Foi example, the three foims of Chicago 
Reading Test D and the Metropolitan and Stanfoid Ad- 
vanced Reading Tests were administered to the same ele- 
mentary pupils It IS planned to determine the equiva- 
lence of the raw scoies and, on the basis of these data, 
to make any necessary adjustments to secure precise com- 
parability of the norms 

i 

t" ~ 


Professoi Karl J Holzinger of the University of Chi- 
cago has wiitten a treatise on Factor Analysis with the 
assistance of Harry H Harman, Research Associate This 
volume IS being published by the University of Chicago 
Press and will be ready this fall Professor Holzinger 
is also joint author of two new monogiaphs on the appli- 
cation of factorial methods The collaborating authors 
are M A Wenger, Frances Swineford and Harry H 
Harman These monographs will also be published by 
the University of Chicago Press 


Wright Junior College m Chicago has recently begun 
a three-year study on the evaluation of terminal educa- 
tion The study is a part of a comprehensive investiga- 
tion of junior college terminal education being cained 
on in nine selected junior colleges by the American Asso- 
ciation of Junior Colleges with a grunt made by the 
General Education Board 

The Wright study will attempt to evaluate the pres- 
ent terminal general and terminal occupational programs 
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offered at that institution One of the purposes of the 
study Will be the development of techniques of evaluation 
foi use byothei schools 

An extensive measuiement progiam will be initiated 
in Septembei foi the incoming fieshman class Measure- 
ments will be made in twelve areas effective thinking 
command of skills and undeislandings in the majoi cul- 
tural aieaSj functional undeistandmg of the basic facts of 
health and disease, mtciests, appieciations, consumer 
competence, occupational efliciency, personal-socio adapt- 
ability, sociO'Civic consciousness, attitudes, worthy use of 
leisuie, functional philosophy of life Several measuring 
devices now available will be used as well as others which 
aie now being developed as a pait of the study Those 
m the foimer category include the CoopeiativeTest Serv- 
ice Geneial Cultuie and Contemporaiy Affairs tests, the 
ICuder Preference Record, and two of the tests of the 
Progressive Education Association on interpretation of 
data and nature of pioof. 

The study is being conducted by a gioup composed of 
Dean William PI Conley, Bernaid Gold, Alice Griffin, 
Max D. Engelhart, and Leland Medsker. 


Soon to be published by the Social Science Research 
Council IS a monograph on the prediction of personal 
adjustment. The text has been prepared by Dr Paul 
Hoist of Pioctei and Gamble The monograph will deal 
with personal adjustment m connection with vocations, 
schools, marriage, and ciiminal recidivism. 
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CUMULATIVE TEST RECORDS THEIR NATURE 

AND USES 


AKTHUK E TUAXLER 
Educational Recofcts Bureau 


M ost schools now lecogmze ceitain values jn ob- 
jective tests of academic aptitude and achievement and 
employ such tests to some extent m the appraisal, placement, 
mstiiiction, and guidance of then pupils It is generally 
Lindeistood that objective tests do not measuie all educational 
outcomes, but studies have lepeatedly dcmonstiated that they 
do measuie ceitam aspects of ability and achievement that aie 
impoitant m the scholastic success of individual boys and 
gills 

It may seem meiely a reiteiation of an obvious point to 
say that the value of a testing program is daectly proportional 
to the natuie and extent of the uses of the lesults by the 
faculty and students Nevertheless, emphasis on this point is 
necessaiy, for not mfieqiiently school authorities administer a 
senes of tests, file the scoies, and then give no further atten- 
tion to the test data When no impiovement is noted, they 
blame the tests when the real fault lies with their failure to 
study the results Tests aie not in themselves remedial mstiu- 
ments, they are tools which can be indispensable aids to diag- 
nosis and thus form an important basis for the planning of 
instruction and guidance, piovrded someone carefully and in- 
telligently studies the data which they provide, The analysis 
of the test results may to some extent be concerned with 
groups, but It should deal primarily with individuals 

Before test results can be studied and used to best advan- 
tage they must be recorded in some convenient form Alpha- 
betical class lists of the scores and percentile ranks of indi- 
vidual pupils, accompanied by sheets showing the distributions 
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of scoies foi various classes, aie veiy useful to teachers for 
piiiposes of a quick suivey of the icsults foi individuals or 
gLOups with Lefeietice to icstiicted areas of ability or achieve- 
ment They do not, liowevei, leadily provide a compre- 
hensive pictuie of the icsults of all tests taken by any one 
individual 

Individual lecoid sheets on which the icsults fiom a single 
testing piogiam aie sumrnaiized in tabiilai oi giaphic foim 
piovide a veiy helpful pictuic of the status of a pupil at any 
one time One can, fot example, admimstei a general achieve- 
ment test batteiy such as the Mctiopolitan oi the Stanfoid, 
plot the achievement piofile of each pupil, and thus obtain 
a giaphic lepieseiUation of sUeiigtlis and weaknesses that 
gicatly simplifies ihc pioblem of diagnosis A giaph of this 
type IS illustiated in Figuie 1 One can see at a glance that in 
compaiison with the giadt nouns, this pupil is stiong in lead- 
ing, litciatiiie, histoiy and civics, and gcogiaphy, hut lelatively 
weak in authmetic and spelling 

However, distiibutions, class lists, and diagnostic piofiles 
lesultmg from one testing piogiam shaie a common limita- 
tion They show stiUus, but iJtcy do not ^how {/lorvth Foi 
both iiistiuction and guidance, the concept of giowth— how 
fai a pupil has come within a ccitain peuod and how far 
he should be able Lo go — is piobably fully as impoitant ns the 
concept of piesent status 

Now, theie is a type of lecoid that provides evidence about 
both status at any testing peuod and giowth between testing 
periods This is the utdmdtial cumulfitive fecoid It Is un- 
questionably the most valuable aid to the intelligent and effi- 
cient use of test lesults yet devised It is to othei kinds of 
iccoids what a motion pictiiie is to a snap shot 

The cumulative lecoid picsiipposes a regulai, systematic 
testing piogram, If tests aic administeied in a school at irieg- 
iilar inteivals and without definite plan, the value of a 
rccoid of this kind will be greatly cui tailed, but even undei 
these conditions, it will piobably piovc more useful than any 
other kind of record of test lesults 
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E\plannlion and Ulus traiwn of Cumtdanve Test Records 
Any recoid foi individual pupils that piovides foi succes- 
sive additions of the same type of data ovei a peuod of 
yeais and thus makes possible a study of the changes that have 
taken place may be called a cunuilaLive lecoid Thus, a tabu- 
lai aiiaiigeiuent of test scoies and pciccntiles by yeais is cumu- 
lative in natiiic Test lesults cnteied in this way, howevei, 
cannot leadily be appiehended quickly, but lequiie detailed 
study This fact has led many peisons to favoi the graphic 
repiescntation of test lesults Among the vaiious types of 
giaphs thus fai devised, the giichron peicentile giaph first 
employed in the Amciican Council cumulative iccoid foims 
stands out as the most widely used type Its success is no 
doubt due paitly to the fact that it will accommodate any 
kind of test lesiilt that can be expiessed in teims of peicentiles 
One of the best known adaptations of the Ameiican Conn- 
ed foim IS the Kducational Rccouls Bnieau cumulative recoid 
foi independent schools This foim, like the American Coun- 
cil foim, is planned for six yeais, but it may be expanded to 
include any numbci of ycais The test poilioti of this type 
of lecoid IS illustialcd m higuies 2, 3, and 4 Let us note in 
some detail the natiiic of the information piovidcd 

The card is divided by heavy veitical lines into bioad 
columns, each of which lepiescnts a giade oi a yeai in the 
life of the pupil The year and giade aic indicated at the 
top of each column 

The fiont of the card is devoted almost entiiely to a rec 
Old of class woik and to an extensive test lecoid Since 
the main puipose of this niticle is to discuss test recoids, the 
poition of the sample foims dealing with subjects, maiks, and 
credits has not been filled out The test lesults aie lepoited 
m both tabiilai and graphic form The scores and coi respond- 
ing percentiles aie eiileied in the table and the peicentiles are 
then used as the basis of the giaph, which occupies approxi- 
mately the lower half of the caid 

The graph of test scoics is the elcaiest phase of the lecord 
to one familiar with graphs of this kind, but it often seems 
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somewhat puzzling to pci sons who have had no expeiiencc 
with It. The pciccntiles along the scale at the left aie placed 
accoiding to standard deviations in a noimal distiibution, and 
thus the distance between successive pcicentiles is much smallei 
neai the median than ntai the cxti ernes The median, oi 
50th peicentile, is marked by the heavy line going hoilzontally 
across the graph The symbols at the top — ^Jy, An, S, 0, and 
so foith — stand foi the months of the yeai The months aie 
grouped accoidmg to the school ycai lathei than the calendar 
yeai 

The same peicentile data that aie shown in the table of 
scores aie entered in the giaph, except that to prevent ovei- 
ciowdiiig, the peicciUiles foi the paits of the English test 
have been omitted fiom the giaph The percentiles used in 
these lecortls aie based on lesults in independent schools, but 
the Intel pi ctation of public-school peicentile latings would 
be made in exactly the same way 

The small dots on the giaph show the placement of the 
vaiious peicentilcs, the dots being identified by the abbieviated 
names of the tests punted ncai them Foi example, in Figiue 
2, the dot towaid the top of the giaph is labeled “Fiench" 
to indicate that it stands for the peicentile on the Coopeiative 
Flench test The peicentile foi the pupil’s total scoic of 61 
on the Ficnch test is 93, and this is indicated by placing the 
dot opposite 93, one of the points shown on the peicentile 
scale at the left of the chait In othei woids, the pupil’s 
Fiench score was above the scoics of 93 pei cent of the mde 
pendent-school ninth-giade fiist-yeai Fiench students who took 
the test in the spiing of 1941, 

The percentile points foi tests that are in the same field 
from year to yeai aie connected by lines, so that one can 
readily follow a paiticiilai type of achievement thioughout the 
whole period coveted by the test For mstance, one of the 
lines in Figure 3 uins fiom the aiithmctic peicentile in Grade 
6 to the arithmetic peicentile in Grade 7, and from that point 
to the arithmetic peicentile in Giade 8, thence to the ele- 
mentary algebra percentile in Giade 9, et ceteia Achieve- 
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iiient peicentiles are connected with solid lines, academic apti- 
tude peicentiles with broken lines, and chionological age pei- 
centiles with dotted lines 

The lecoid shown m Pigiiie 2^ that of Edwin Mai tin, ^ 
coveis only one yeai In many independent schools, a consid- 
erable piopoition of the lecoids will be ot this type, foi the 
number of one-yeai students attending piivate schools tends 
to be faiily laige Even in the case of single-yeai lecoids, 
it IS desiiable to record the data on a cumulative lecord foim, 
foi such a proceduie facilitates compaiisons between academic 
aptitude and achievement and makes it possible to summaiize 
leadily the student’s test lecoid foi the year as a whole 
Figiue 2 shows that in the fall of 1940, Edwin was close 
to, but slightly below aveiage foi his giade in cluonological 
age and that he was somewhat below the median m academic 
aptitude, as indicated by the lesiilts of the Ameiican Council 
Psychological Examination, and m leading, as measiucd by 
the Nelson-Denny Test These lesults aie lecoided diiectly 
below the lettei 0, which shows that the data weie obtained 
in Octobei 

The spiing, 1941, percentiles aie cntcied beneath the let- 
tei /i, and thus one knows that the tests weie given in Apiil 
Edwin seems to be an able student of foieign language As 
alieady indicated, his French scoie was above those of all but 
7 per cent of the fiist-yeai Ficiich students in Giade 9 His 
total scoie on the Latin test fell within the highest thud of 
the independent-school nmCh-giade fiist-yeai gioup 

In science and elementary algebia, the boy was above the 
independent-school median but not outstanding His total 
English peicentile and his hteiaiy acquaintance pcicentile weie 
below the median but above his academic aptitude peicentile 
In general, Edwin’s achievement test peicentiles weie some- 
what highei than his peicentiles in academic aptitude and 
reading This is, of couisc, an encotnaging finding, for it 
indicates that, piesumably because of application and hard 

^Thcae are actual lest rccoids, but the names of the pupils and the schools 
arc hctitious 
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work, the boy’s iichievemcnt record neui the end of the school 
year was better than one would expect it to be on the basis 
of the fall test lesults 

Let ns now exnininc a ciimiilntive lecoid coveimg several 
yeais The test lecoid of Betty A Stetson, as shown m Fig. 
lire 3, IS that of a gul who was a little yoiingei than the 
Aveiage pupil in het giade but who was geneially high in both 
academic aptitude and achievement liei Otis intelligence 
test peicentiles in Giade 6 wcie exceptionally high Hei 
later academic aptitude peicentiles weic a little lowei, but all 
of them wcie significantly above the median foi hei giade In 
fact, hei Otis scoies in Giades 6 and 7 and hei scoies on the 
Ameiican Council Psychological Examination in Giades 9 and 
10 weie in the highest tenth of the scoies made by the nule- 
pendent-school pupils at the same giade levels, 

Betty’s achievement lesL peicentiles weie, in geneial, some- 
what lower than hei peicentiles in academic aptitude, but most 
of them wcic in tJic uppei half of the scoits of the pupils 
in her giade The only achievement peicentiles below the 
median weic those foi spelling in Giade 6, geogiaphy in 
Glade 7, arithmetic in Giade 8, geneial science m Giade 10, 
and modern European histoiy in Giade 11 The histOLy scoie 
was the only vciy low icsiilt in the entile lecoid 

This gill IS obviously an excellent rcadei On the leading 
tests, she maintained a position within the highest tenth of her 
grade thioughoiit the entiie peiiod 

In a giaphic lecoid of this kind, giowth in any subject 
piecisely equal to the growth of the gioup as a whole in that 
subject is shown by an exactly hoii^ontal line That is, if a 
pupil improves just as much as the gioup impioves in a yeai, 
he will maintain the same peiccntile lating fiom one year to 
the next Lines which go upward, then, indicate gieatei than 
average giowth and lines which slope downward suggest less 
than aveiage growth, In interpreting such variations, how- 
evei, one should keep in mind the fact that eveiy test involves 
a ceitain amount of sampling eiioi and that the population on 
which the percentiles aie based Is not exactly the same from 
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year to yc.ii So some vaiiation m peicentiles foi the same 
subject IS to be expected even when giowth is noimal One 
should also icmcmbci that tests of cIifFeient subjects in the 
same held — foi exiimple, algcbia and plane geometiy oi biol- 
ogy and chemistiy — involve somewhat diffeient abilities In 
these instances, changes in peiccntilc latiiig should not be 
inteipietcd in teinis of gjowth 

Hovvevci , maiked gains oi losses in peicentiles on the same 
test, such as an Faighsh test oj a foicign language test, are 
symptomatic, and they may be indicative of a need foi coun- 
seling attention in oidei to find the leason foi the vaiiation 
Foi example, the steady downwaid tiend of Betty’s per- 
centiles in Flench would seem to leqimc an explanation that 
cannot be obtained fiom the iccoid itself 

On the whole, the scveivyeai lecoid shown in Figuie 3 
indicates definitely supciioi ability and achievement A coun- 
seloi, school piincipal, oi college admissions oflicei familiar 
with this type of lecoid could decide almost at a glance that, 
as fai as apLitudc and attainment aic concerned, this girl 
should do well even in a highly selective college 

The iccoid of Chailes W I^oiing, shown in Figiue 4, also 
includes Giades 6 to 12, inclusive, but it covcis a period of 
eight yeais lathei than seven, since this hoy lepeated the 
eighth giadc The gcncial level of the percentiles is in marked 
contrast to that of the peicentiles on the pieceding lecoid 
This IS an ovei-age pupil who is lathei low in both academic 
aptitude and achievement in compaiison with the aveiage for 
his giade. Because he is advanced in chionological age, the 
percentiles conespondmg to mental age and law scores on 
intelligence tests tend to be somewhat higher than the per- 
centiles foi IQ, but with one exception they aie below the 
independent-school median 

In Grade 6, all but one of Charles’ scoies on the New 
Stanfoid Achievement Test weie distiibuted below the median 
foi independent-school pupils at that giade level The next 
yeai, most of his achievement test peicentiles went upwaid to 
some extent, but the peicentiles foi language usage and aiith- 
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mfitic weic the only ones above the iiidependent-school median 
foi Giade 7 In the following yeais, neaily all his peicentiles 
weie below the median loi his gtacle, although some of them 
weie not far below it. 

Theie is some evidence that this boy had lead lathei 
Widely and that he was a fauly competent leadei Three 
of his liteiatuic pciccntdcs and tluec of ins leading peiccntiks 
wcie above the median foi his gi.ide 

Chailes' lepctition of Gi.adc 8 did not significantly laise 
his subsequent lecoid on the achievement tests The only line 
that went upwaid noticeably as a icsult of the lepctition of 
this giadc was the one foi chionological age The whole 
rccoid IS that of a pupil who piobably should not attempt 
to entci the usual libcial aits college aftei giaduation from 
the sccond.uy school Rathei, he needs guidance into piepaia 
tion for some type of vocation the demands of which me 
consistent with his mcdiocic scholastic attainments 

It will be obscivcd that theic is a gencial consistency m 
the test icsults thioughout both illustintive iccoids The gill 
whose lecord is shown in Figiiie 3 was high in academic aptl 
tilde and achievement in the clcmcntaiy school and she main- 
tained this SLipciioiity thioughout the sccondaiy school The 
boy whose lecoid is contained in Figuic 4 was low in academic 
aptitude and achievement in the elemental y school and this 
low rccoid was continued in the secondaiy school In both 
cases, the gencial level of the peicentile latmgs m Giade 12 
could have been piedictcd fiom the lesidts of the achievement 
tests taken in Grade 6 

The tendency of the cumulative lecoid of test results foi 
an individual pupil to be in agi cement from yeai to year is 
one of the most notewoiihy aspects of this type of record 
This tendency is veiified by hundieds of such lecoids which 
have been prepared at the Educational Records Bureau and 
other institutions While the percentiles on an occasional test 
may vary tnaikedly in successive years, the whole picture of 
a pupil’s record tends to remain much the same This is 
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usually tuie legaidless of tianbfei from one school to another 
oi variation m type of instiuction Comprehensive test rec- 
01 ds obtained even as low as the second or third giade often 
pi edict with lemaikable fidelity the level of achievement a 
pupil Will attain in his senioi yeai of high school The fact 
that test lesults distubuted ovei a long period of time tend 
to be positively con elated causes cumulative test histones to 
have exceptional potential guidance values 

Notwithstanding the general tendency just indicated, it 
IS tine that a pupil’s test recoid foi an entire year may some- 
times be decidedly out of line with his scoies in other years 
When this happens, an explanation that can be made only m 
the light of much othei infoimation about the pupil is re- 
qimed Consequently, a cumulative tecoid of othei kinds of 
data IS needed if one is to make an adequate mterpietation 
of the test lecord Foi this leason and many other reasons, 
It IS advisable foi schools to maintain cumulative recoids that 
cover not only test lesults but that include home backgiound, 
class woik, inteiests and activities, peisonality adjustment, and 
vaiious othei factois The inteiielationships of the different 
kinds of infoimation that can be recorded on a form similai 
to the Ameiican Council caid aie brought out by the recoid 
for Hariy Connelly, as shown in Figuie S 

It IS evident fiom Figure 5 that this boy tended to be below 
the independent-school median in his scoies on the Metio- 
politan Achievement Test taken in Giade 8, but that he was 
consistently above the median m scoies on all the achievement 
tests taken in Grades 9 to 12 He was especially high in 
English, literal y acquaintance, and science The boy’s superior 
test record in the foiu high-school grades agices with his con- 
sistently high percentile latmgs on the academic aptitude tests 
It appears that an explanation of the maiked difference 
between Hariy’s test scoies in Grade 8 and his test scores 
m the later yeais is to be found in the data entered on the 
back of the caid Although he was obviously bright, he was 
lazy and disoideily m the eighth grade and it is probable 
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thathe lacked both the piepaiation and the mteiest m the test 
Itself that would be lequiiecl foi high scoies on the Metro- 
politan test, 01 any othei test of geneial achievement Theic 
was much impiovement m the boy’s behavioi and attitude 
m the ninth giade and m subsequent glades, and consequently 
his achievement ijici eased until it was piopoitionate to his 
ability The whole pictuic is that of an intelligent, able boy 
who was immatuie in behavioi and attitude but who became 
much moic matuie dining the sccondaiy-school yeais At the 
end of the secondary school, he unquestionably had the ability 
and the piepaiation foi bettei than aveiage college work 
Experience indicates that a lecoid of this kind fninishes a fai 
bettei basis foi piognosis of college success than is piovided 
by a tiansciipt of ciedits and an admission foim filled out 
by the school when the pupil is neai the end of his secoiidaiy- 
school cDuise 

Cumulative lecoids in teims of percentiles aie not the 
only kind of graphic lecoid of test icsults that can be used 
If the results of all tests employed m a school's piogram 
aie expiessed in teims of standaid scoies, Scaled Scores, or 
some other comparable unit, the data may be giaphed on that 
basis Such units aie sometimes piefeiablc to peicentiles for 
pLiiposes of showing growth and if they take their ougin from 
a common standai dilation gioiip, as do the Scaled Scores 
of the Cooperative Test Seivice, the influence of the selective 
factoi found in ceitain subjects, such as the foreign languages, 
IS obviated ^ 

It should be cleaily undei stood that cumulative records 
of test results can be kept without the use of any graph what- 
soever The prepaiation of the graphic pait of the lecoid is, 
of coiiise, a time-consuming cleiica! job While it is a dis- 
tinct aid to interpretation, schools in which the time and cost 

2For an iHuatration of a cumulative record based on Scaled Scores see John 
C Flana^nn, The Caoperaiwe Achievement Tests A Unlletm Reporting the 
Rasie Pi inciples and P/oeeduies Used in the Development of Their Sj/slem of 
Scaled SeoreSj p 37 New York The Cooperfitivc Test Service, December, 1939 
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of the giaph ^vould be prohibitive can maintain usable test 
lecoicls in tabiilni foim The mam thing is to lecoid the data 
in oigani/cd fashion so that tiends can be discerned 

Uses o[ Cvwnhtive Test Records 

As alieady indic.iled, the uses that aic made of cimuilatlve 
test lecoids depend hugely on the mtcicst^ initiative, and 
iindei standing of the administiatioii and faculty in each local 
school Among the possible uses of such recoids aie the 
following • 

1 Counselois may use cumulative test lecoids m con- 
ferung with pupils and guiding them lowaid educational and 
vocational choices consistent with theli ability and achievement 

2 Teacheis may study them m oidei to plan their instruc- 
tion to accoid with the aptitude, knowledge, and iindei stand- 
ing of the individuals in then classes 

3 Admmistiatois and pcisonnel officcis may lefer to 
them when coiifeiung with paicnts about their childien® 

4 Puncipals and guidance dhectois may take them into 
consideration when leconimcnding giaduates to colleges or to 
prospective employers 

5 College admissions ofliiceis may use them as one type 
of evidence on which decisions about admitting applicants are 
based. In oidei to conscive the time of the college admis- 
sions olficei, the school should of coiuse include a paiagiaph 
of interpretation when the record is sent to the college Ad- 
missions ofliceis expect to leccive fiom the school an estimate 
of a candidate's fitness and they will place more ciedence m 
the estimate when it is based in part upon tangible infoimation 

6 Schools may employ them in placing tiansfei pupils in 
coiiiscs to which they aic suited 

7 Administi alive ofliccis and depaitmcnt heads may use 
them in sectioning classes on the basis of ability 

®A»i excellent diiicussicii of tine type of use la givcii in Robert N, Hilkert, 
"Parents and Cumultitivc Recoida," Educatioiui] Eecofd, Supplement No 13, pp 
172-83 Wnshingtoii, D C, Ameuenn Council on Educntion, January, 1940 
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8 Remedial teacheis may consult them in selecting pupils 
foi special lemeclial woik and in planning that woik 

9 Psychologists and psychiatrists may turn to them foi 
leads in diagnosing peisonality maladjustments and planning 
tieatment 

10 Supeiintendeiits and principals may make limited use 
of them in appraising the work of the school and introducing 
modifications This type of use should be carefully thought 
out and cautiously applied 

1 1 CoLinseJois and teachers may refer to them as a means 
of stimulating pupils to do their best work This is a legiti- 
mate use if the comparison is diiectly with the previous record 
of each pupil and only mdiiectly, or not at all, with that of 
other pupils ^ 

12 Finally, the entire faculty may employ cumulative test 
records in developing what is perhaps the school’s most im- 
portant function — the planning for each pupil of a program 
that is suited to him and the individualisation of instruction 
in accordance with such a piogiam ^ 

The Ameucan Council cumulative record forms, from 
which the card used In the illustialions in this article was 
adapted, are now being revised” A tentative draft of the 
levised high-school form is ready and it will be tried out soon 
in several publrc high schools Changes have been made m 
various parts of the record to accord with modern trends in 
education It is significant that in the levised form the test 


^Thc use of test recorda in pupil self-appraisal is described m Richard D 
Allen, Self-Mcasu) emejit Ptojccit in Group Guidance, Inor Gioup Guidance 
Senes, Volume III (New York Inoi Publishing Compmy, Inc, 1934), xviii-|- 
274 

^Sec Ben D Wood, “The Need for Comparable Measurements in Indi- 
vidualizing Education,” Educaltoiial Record, Supplement No 12, pp S 13 
Washington, D C AmencTn Council on Education, January, 1919 

^The revision of the American Council cumulative record forms is being 
done by a subcommiitee of the Committee on Measurement and Guidance of 
the American Council on Education The chan man of the subcommittee is 
Eugene R Smith, Beaver Country Day School, Chestnut Hdl, Massachusetts 
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section continues to be one of the most impoitant aspects of 
the lecoid Any fonvaicl-lookmpr cumulative iccoid, legmd- 
less of wbethei it is devised by ,m oigani/ation of national 
scope 01 by a local school system, will inevitably include a 
thoiough test lecoid, foi it is becoming geneiaJly lecognized 
that .1 piciequisiLe to an adequate piogiain of guidance is a 
Lonipiebcnsivc, systematiL testing piogiam 
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T he objectives of the student counseling progiam 
at the UniveiMty of Minnesota have not changed funda- 
mentally since its inception in the Aits College m 1923 The 
aims of the gionp of Aits College faculty coiinselois as stated 
111 1928 by Pateison weie "Fust, to bung about a more 
haunonious adjustment ol individual students to the oppor- 
tunities available within and without the Univeisity, and sec- 
ond, to establish, as far as possible, a friendly and constructive 
relationship between individual membeis of the faculty and 
students desiiing such contact” (2 265-266) 

Subsequent tiends m the Univcisity’s curricular oiganiza- 
tion and the developing facilities foi peisonnel woik have led 
to a gicatei diffeientiation of function within the total coun- 
seling progiam- The inci easing complexity and the conse- 
quent piofessionalization of ceitain types of counseling re- 
sulted 111 the establishment of the University Testing Bureau 
among othei specialized agencies foi the ticatment of student 
problems This Univeisity-wide counseling agency is both 
coordinate and cooidiiiated with the counseling agencies of the 
sepal ate colleges within the University 

^AasisUDce in the tabulation and summarization of materials was provided 
by Minnesota work projects under pioject 671+, sub-project 85, sponsored by 
the University of Minnesota 

^For an historical treatment of these developments see E G Williamson 
nnd T K Sabin (Minneapolis Burgess Publishing Company, 1940) 115 pp 

341 



LDUCAHONAL and rSYCtlOIOGlCAL MLASUREMLNl 


The Testing Bui can, in its function as a counseling 
agency,^ piovicles piofcssionali/ed educational and vocational 
guidance supplemental y to the sei vices of othei peisonnel 
clepaitments on the cninpii*? Counseling is pei’toitned on 
m individiKiliycd basis, the coiinseloi using infoimation from 
tests, lepoits fiom othci peisonnel woikeis on the campus 
and from community .md high school agencies, and hom 
clinical inteivicws with the student 

In a senes of papeis published lecently, the authorg 
ticatccl the pioblcm of the evaluation of these counseling 
sei vices The fiisl in the senes piesented a systematic anal- 
ysis of expcumcntal methods as applied to this type of coun- 
seling (^) Based upon oiu conclusions with icgaid to 
method, two expciinicnlal evaluations of this counseling weie 
lepoited 

The fust of these expciimcnLs (5) investigated the lela- 
tive adjustment ot students who did and did not coopeiate 
with the counscloi The ciiLeii.i of adjustmciU and coopeia- 
Uon wcie judgments by woikcis who had not been involved 
in the counseling piocess and weie based on leadings of the 
case histoiy and follow-up intei views The lesults showed 
that students who coopeiatcd wcic nioie likely to he adjusted 

The second cxpciiment (6) tested the hypothesis that 
students couiibeled by the Buie.iu would he bettci adjusted 
and moie successful academically than students who had not 
been counseled by the Buicau oi any college counseling agency 
This hypothesis was found to hold foi the compaiison of a 
counseled with a matched iion-coiinscled gioup of fieshman 
Arts College students The ciiteiia in this study were judg- 
ments of adjustment and coopeiaiion and aveiage giade 
achievement. 

Futiuc piogiesb in counseling of this natiue will depend 
upon knowledge of the lesoiuces and techniques utilized by 
the counselors, the types of pioblems dealt with, and the 
effectiveness with which these problems weic handled, In 

^The Bureau also functions as a Univcisiiy-wide testing agency and as a 
locus for rescaich in icsiing and counseling 

342 



ANALYTICAL DESCRIPTION OF SlUDENT COUNSELING 


this papei we aie conceined with giving a lepiesentative pic- 
tuie of the lesoiirces and geneial techniques utilized in the 
Univcisity Testing Biiieaii ovei the peuod hom 1932 to 
1935 

Analyses of faculty counseling in the Arts College (3, 8) 
and an exploiatoiy analysis ot Testing Biiieau counseling 
(4 253-260) foim the backgiound foi the piesent study 
The exploiatoiy study of Buieau counseling was based on a 
sampling of 196 student cases, analyzed as to oiigin, class, 
and college The lepiesentatlveness of these cases in teims of 
high school scholaiship and college aptitude scoie was deter- 
mined Summaiies weie piesented of the kinds of case data 
used, the agencies consulted oi lefeiied to foi diagnosis and 
treatment, the types and fiequcncies of student pioblems, and 
the geneial counseling techniques used 

The piesent study is designed to amplify the desciiption of 
Buieau counseling from a much bioadci sampling of the 
total case load A total of 2053 student cases, the bulk 
of students who came in foi complete counseling services over 
the period fiom 1932 to 1935, foimed the population for 
this suivey * The actual case hisfcoiy folders, including rec- 
01 ds of counseling intei views, weie analyzed, and the piesence 
of certain items of infoimation tabulated No questionnaiies 
filled out by the students were used foi this analysis This 
study, theiefoie, piovides an answer to the question, “What 
IS counseling'?” in terms of the judgments of counselois made 
111 terms of paiticulai students and not of students in general 

Of the 2053 cases, 1223 students were men and 830 
weie women Classified according to year m college, theie 
weie 617 pie-college students (lecent high school giaduates), 
721 freshmen, 482 sophomoies, 143 juniois, 54 seniois and 
36 graduate students By college the distiibutions were Gen- 
eial College, 428, Aits College, 1038, pie-college who did 
not matriculate In the Umveisity, 197, Chemistry-Eiigineei- 

^By "complete cQunaeling servicea" is meant testing and extensive inter- 
viewing The Buieau also provides many types of testing aersices for members 
of the student peisonnel staff of the University 
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ing-Miiies, 133, Agiicultuie, 79, Education, 48, Business, 34; 
Medjc.il-Dcntal-Phaimacy, 23, Giaduate School, 36, Niiis- 
ing, 18, and Unweisity College, 5 

Ongin of ihe Cases 

The eflicicncy of a counseling piogiam must, in part, be 
measuiccl on the basis of its integiation and cooidmation with 
othei peisonnel functions Tins ciiteiioii implies that coun- 
selois at vaiious levels of specialization aie aware of the 
limits of then lunctions and aie making use of the seivicei 
of specialized peisonnel woikcrs tliiough the medium of 
leferial 

With this in mind the oiigiii of the cases counseled iti the 
Buieau becomes pcitincnt to its eflicicncy Obviously, the two 
main categoues of ongiii aic lefeued and voluntaiy Over 
fifteen ycais of expeiience in the counseling piogiam at Min- 
nesota have shown that the best lesiilts can be achieved when 
the student comes voluntauly to the counseloi oi seeks assist- 
ance at the suggestion, but not command, of some member of 
the Univeisity staff or student body Of the total of 20S3, 
1069 of the students were classified as voluntaiy cases Actu- 
ally, of the 984 lemaming students classified as icfcired cases, 
ill only 93 cases was icfciial made by University officials 
in the spiiit of piessuic These weie students of low scholai- 
ship who had been lefeiied to the Bui can as pait of pro- 
cecluies involved in scholastic discipline A total of 791 stu- 
dents had been lefcucd to the Buieau foi testing and coun- 
seling aftci mtei views with a college counseloi oi faculty 
niembei In addition, 100 students had been lefeiied by high 
school counseloi s oi community wclfaie agencies 

The laigest pioportion of the voluntaiy cases, 892, came 
to the Buieau aftei having heaid about its seiviccs through 
bulletins, class lectmes, fiicnds, oi i datives In addition, 122 
students weie told about the Biu can’s scivice in the Regis- 
trai’s office and 55 had learned about it fiom high school or 
college teacheis other than counsclois, 

What distinguishes these students coming by way of vari- 
ous campus and community agencies? Analyses of thf vari- 
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ajice (1 Chap V) in high school peicentiles and college apti- 
'tude test scoies give a paitial answei to the question Re- 
feiied students tend to be lowei than voluntaiy students in 
both high school achievement and college aptitude (F values 
of 24 08 and 76 59, lespectively , both well beyond the one 
pci cent point) While theie aie significant vaiiances be- 
tween lefeiied and vohintaiy cases, "t” tests (1 97) le- 
vealed that the vaiiation of the sub-categoiies in college apth 
tilde was homogeneous within each of the two main divisions 
This means that there weie no leliable diffeiences m ability, 
eithei among students who had been lefeired by a faculty 
coimseloi, a college official, oi a high school counseloi oi 
welfaie officei, oi among students who came voluntaiily as a 
result of contact with high school oi college faculty, the Reg^ 
istiai’s office, oi some infoimal souice of infoimation about 
the Buieau’s sei vices 

In the case of high school achievement, diffeiences do exist 
between students refeired by high school counselors or wel- 
faie agencies and students lefeired by college officials, college 
counselors, oi faculty membeis The lelations found between 
type of pioblems and oiigin of case tend to clarify the pic- 
tuie Students who have been lefeiied by high school conn- 
seJois or welfaie agencies aie more likely to have financial 
and health pioblems and aie less likely to have vocational 
and educational problems Thus we may conclude that stu- 
dents lefeiied fiom these souices outside the Univeisity aie 
likely to be students with the financial oi health pioblems 
leferied because they were good students in high school and 
had well-developed vocational goals 

The 1 elation of type of pioblem to origin of case also 
gave indications that the othei two types of refeired students 
tend to have fewei financial pioblems and that students who 
came voluntarily weie more likely to have vocational pioblems 
and Jess likely to have health problems 

Types and Frequencies of Problems 

Before more adequate evaluations of counseling can be 
madCjithe types and frequencies of problems encountered must 
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be descLibecl Then only can the fonnclation be laid for more 
piecise expel imuitation in which ticatment methods aie dif- 
ferentiated accoiding to theii lelative value for vaiions types 
of pioblems oi problem constellations 

The scope of this papci is limited to a geneial description 
of coimsehng Anothet paper Is being piepared which will 
desciibe how these pioblems clustei foi the students counseled 
and with what chaiacteiistics these chisteis aie absociated 
This IS conceived as the fust step towaid evolving a symptom- 
atology foi use in counseling 

TABLL 1 


lYPCS Oil srUDrNl PIIOULPMS as arCORDLD IN CASL IIIsrOIULS 


A 


Frequency 

of 

Occurrence 


Financial 

I Need 01 desire foi p.iit time work, scholarship or 

loans j inadequate finanLcs 443 


U. Vocational 

1 Poor aptitude for chosen vocation 270 

2 Inability to decide between two or more vocational 

choices . 750 

3 Definite choice but wants coiifirmatinn oi cncoui- 

agement . . . 976 

4 Definite choice but 111 doubt about aptitude 116 

5 Definite choice but based only on inniicncc of 

faunlyi friends, etc . 108 

6 Dearth of interest in any vocation 42 

7 Information needed about occupations m general 66 

8 Vocational choice without adequate self-analvsis 60 

9 Inadequate infoimation in regard to professional 

choice , . 50 2444 


C Educational 

1 Poor aptitude foi college wotk ■ . 292 

2. Selection of couisc in line with occuiiational choice 632 

3 Infcrioiity in academic skills such as leading, study 

liabits, English usage, etc, . 446 

4 Understanding gindiiig standaids, . 15 

5. High general aptitude and poor scholastic achieve- 
ments ... , . 184 

6 Understanding responsibilities in college 134 
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7 High aptitude hampered by standard curiicula 20 

8 Outside work inteifenng with studies 71 

9 University entrance witliout piopei lequircments 48 1842 

D Social, Personal, and Emotional 

1 Too much social life or too many social activities 43 

2 Inadeciuate participation in extia-curricular activi- 
ties 23 

3 Selecting student activities ni line with interests 8 

4 Social personality traits which may hinder piofes- 

sionnl success , 115 

5 Need for encouragement and self-confidcnce 230 

6 Social timidity 89 

7 Emotional disturbances 172 

8 Family domination in vocational choice 78 

9, Conflict with family oi friends 91 

10 Parental anxiety for a wise vocational choice 52 

11 Fcai of intellectual inadequacy 34 

12. Idealization of a profession 17 

13, Ovei -evaluation of a college degiee 17 969 

E Health and Physical Disabilities 

1 Seuous physical disabilities 107 

2 Easily fatigued 27 

3 Inability to do justice to woik because of intermit- 
tent illness 35 

4 Physical habits, diet and sleep, etc 9 178 


5876 


Table 1 shows the distribution of the 5876 problems found 
in the case lecords of these 2053 cases We see that about 
two-thirds of the problems of the students were of an educa- 
tional or vocational natuie The most fiequent vocational 
pioblems weie found among cases of students unable to decide 
between two or more vocational choices or who wanted con- 
firmation 01 encouragement in making a vocational choice 
The most typical educational problems weie those of selecting 
a training piogram appropiiate to the vocational choice and 
those due to uifeiioiity m such academic skills as teadmgj 
study habits, and English usage 

Social-personal-emotional problems were the next most 
frequent types of problems Modal snb-types were less marked 
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heic The three most fiequcnt types of sociahpeisoiial-emo. 
tioHEil pioblenis weic need foi cncouiagemcnt and self-con^ 
fidence, soci.il pcisonality UaiLs which may hmdei piofessiotial 
success, and cmotiojial distuibances, foi the most pait of a 
iion-psychiritiic mitLiic 

Health piohlcms wtie inficquLiitly found among these 
cases, 178 piohlems being discovcicd and lepoited to the 
Bill can by the Univcisity Student Health Seivice 

The Infoimahonal Basis of Counseling 

The progiess of counseling as a discipline has been char* 
acterized by a departiiie fiom ^‘gold buck" methods of judg- 
ing abilities and cliaiactci The ticnd is towaul a greatei 
lehaiicc upon the systematic collection of data about the indi- 
vidual by means of standaidizcd tests, repoits fiom othei 
people who have had contacts with him xiiidei divcise condi- 
tions, medical iccoids, and so on The only vestige of early 
coimbcling methods is the Intel view This is still the most 
vital paiL of the coimsolmg piocess, but is now the melting pot 
in which the btudent and the counseloi integiate information 
to diaw out a unified picUuc of the individual and to plan the 
next steps in adjustment 

In Tabic 2 we picscnt a summaiy of the niimbci and fie 
qucncies with which each soiuce of data was consulted by the 
counseloi If ficqiiency and souice of data aic giouped to- 
gethei, 27,866 units of data weie used as the basis foi coun- 
seling The most ficqucnt souicc of information was voca- 
tional and educational tests given in the Testing Bureau 
Cleaiance slips fiom the Faculty-Student Contact Desk (6 83) 
piovided information in 2038 cases 

Othci impouaiit souices of data weic Health Service 
lepoits, Univcisjty Fntraiice Test lating, and giades fiom 
the Registiar’s oflice The fact that leports from family or 
other lelatives weic the least frequent souices of infoimation 
could be taken as an indication of the need for a study to 
determine whether a social worker would be a valuable addi- 
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Total of 
each Type 

of Data 

Collected 1253 5612 1642 14jS74 405,3 133 11 365 9 10 174 27,fl66 
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tioii to the Testing Biire^iu staff It 'vvoiild be necessary to 
detcimme how much infoimation would be added and how 
useful that infoimatioii would be in diagnosis and tieatinent 
Counseling Pioicdttics Classified hy Type of Pioblem 
One of the aieas foi icseaich in counseling which has 
been least exploited is the piecisc dcsciiptioii of the counsel- 
ing inteiview — the iclatioiiships between vaiious counseling 
pioccsscs and the ellectivencss with which each type of prob- 
lem IS handled The ultimate objective of such desciiption is 
the delineation of these counseling piocesses in terms of 
fimdamental psychological categories One possible prelimi- 
naiy step may be the geneial dcsciiption of what the coun- 
seloi did with rcgaid to vanous types of problems This is 
called a geneial dcsciiption because it docs not attempt to 
Lake into account the psychological setting within the mtei- 
view (eg, the specific attitudes the counselor and counselee 
had toward each othei at that point) when the behavior 
desciibed occuiicd 

In Table 3 we picscnt a geneial dcsciiption of the 
Bureau’s counseling pioccdiues 'riic data show that the com- 
monest procediiics la counseling students with financial prob- 
lems took the foini of discussing the need for woik, discussing 
scholarships and loan funds as a soiucc of money, and dis- 
cussing the iclation of pait-Lime work to the student's class 
schedule 

In the ticatmcnt of vocational pioblcms, the couiiselois 
jelled mainly on discussions of aptitude and on advice and 
recommendations of occupational choice on the basis of test 
results Other frequent procedures include advising voca- 
tional “tryouts" through college coiuses, desciiptions of occu- 
pations and advising a geneial background training before a 
definite choice is made. 

With educational problems, the most frequent procedure 
was aid in selecting a schedule of classes in line with aptitude 
In another large number of cases the counselors discussed 
comse prerequisites, sequence of coiuses, find the like 
Attempts at cultivation of interests in studies and scholastic 
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TABLE 3 

COUNSELING PROCEDURES CLASSIFIED DY q'YPIi OF PROBLEM 


of 

Types of Problems Occurrence 

Financial Ptoblems 

1 Discussion of relation of pait-time work to class sched- 
ule 71 

2 Discussion of need foi work 83 

3 Suggestions of ways of getting jobs 12 

4 Discussion of student’s expenses and financial resources 54 

5 Discussion of scholaisliip and loan funds 78 

6 Letters of recommendation for jobs, scliolaisliips and 

loans 40 

7 Refeiial to emplovinent bureau 22 

8 Rcfenal to financial aid agencies 31 391 

V ocat tonal Problems 

1 Desciiption of occupations 232 

2 Rcfenal to infoimational books 128 

3 Discussion of aptitude , 1120 

4 Discussion of student's financial resources foi occupa- 
tional training , 125 

5 Vocational tiyouts through college courses 351 

6 Advice and recommendation of occupational choice (on 

basis of test results) , 1346 

7 Advice of geneial backgiound tiaming before definite 

choice is made 251 

8 Discussion of method of entering and securing employ- 
ment in chosen occiioation 120 3673 

Educational Ptoblems 

1 Use of class schedule foi piogram making 4 

2 Discussion of course pierequisitcs, sequence of courses, 

etc 304 

3 Cultivation of mtciests in studies scholastic recoid, etc l?l 

4 Evplanation of lecitation method of study 13 

5 Disciictiinn of special suitoiindings conducive to effec- 
tive study 31 

6 Discussion of methods of vorabiilaiy building 27 

7 Tu tonal aid with specific subicets 20 

8 Aid in selecting a schedule of classes m line Mitli apti- 
tude 642 

9 Aid in budgeting liouis foi study 49 

10 An nttemot to analyze cause foi difficulty with a 

snecific subject 38 

11 Evnhiiation of student’s low aptitude as cause of low 

<;cholarship 34 

12 Recommendation of non-college h^pe of vocational train- 
ing , 1^^ 


I'.nUCAlJONAI AND I’bYCHDJ OGICAI MLASUREMENT 
TAHLE 3 (Continued) 

Erequenev 


Types of Prohlc ms Occurrence 

n Attempt to diagnose aiul ovcicome special disability in 
spelling, giammai, malhematicij, etc 25 

n Discussion .ind explanation of student's icsponsibility 
in college, glad mg stand.uds, etc, 74 

15 Rccommciul student change course of study 182 

16 Discussion of eligibility foi piesciibed work 97 

17 Refeuiil to "How to Study" instiiictors 62 1872 

Social, Pci&oiial and Emotional Pioblems 

1 Warning of over-ciiipliasis put upon his social activities 13 

2 Ai ranging contact with pioper activities such as band 

and debate, etc 1 

3 Suggestion cif piopci activities to tryout foi cxtia-cui- 

iiculai interests , 13 

4 Establishing fiicndly contact with faculty for futuie use 22 

5 General disciissionj encouiagement and assistance with 

pjoblcm of self-con fide nee , 155 

6 Suggested ticntmcnt foi specific pcisonality difficulties 28 

7 Discussion of meeting people and making fi lends 20 

8 Ticatmcnt suggesting special things to do It 

9 Suggesting discussion between family .nul friends ovci* 

mutual conflicts . . 25 

10 Letter 01 Intel view witli members of family over con- 
flicts . » 28 

n Discussion of worries and other emotional problems 108 

12 Advise tiansfcr to anotliei school because of home 

envuonmciTt . . 12 

13 Recommend welfare agency to assist student 2 

14 Rcfeiial to Y M, C A fen aid in social adjustment 

of student , 19 

15 Referral to psvchiati ist foi diagnosis, 53 

16 Referral to Speech Clmic for diagnosis and treatment 

of speech difficulties. 31 ^41 

Health and Physical Disabilities 
! Di'^cussion of liandicnps , . 103 

2. Advising icmcdial gym . 8 

3 Discussion of living arrangement sleep, diet, etc 4 

4 Athletics suggested for bcttci health 2 

5, Referral to child health chnic , , , 1 

6 Referral to Health Service foi special Iicaltli examina- 
tion , . ... 2 122 
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recoids, i ecommeiidation ot a noii-college type of vocational 
tjajjung, and i ecommeiidation that the student change his 
course of study weie othei pioceduies with high fiequency 
The tabulation of the piocedmes used with social-pei- 
sonal-emotional pioblems indicate that the counseloi lelied 
on either a general discussion for enconiagement and assist- 
ance With the problem of self-confidence oi discussed woiiies 
and othei emotional pioblems The next most fiequent step 
was to lefei the student to the psychiatiist foi diagnosis 
Discussion of the physical handicaps involved was by fai 
the mam method used with health pioblems The counseloi 's 
discussion did not impinge upon medical advice but lathei 
upon the lelationship between physical condition and educa- 
tional and vocational adjustments 

Suvmaf y 

A general desciipt/on based on 2053 cases waspiesented 
as a basis foi analytical desciiptioii of counseling m the 
Testing Bureau of the Univeisity of Minnesota ovei a foui 
yeai peiiod fiom 1932 to 1935 inclusive 

This desciiption enables us to see how well the Bureau’s 
counseling service is cooidinated with the geneial personnel 
piogiam of the Univeisity By bioad delineations of pioblem 
aieas handled, of sources and amounts of data used and of 
pioceduies followed, the aiithois hope to bieak giound foi 
a much needed basic desciiption of the psychological piocesses 
involved in counseling intei viewing conducted in a non- 
psychiatiic guidance clinic for college students 

What IS needed is like desciiptions by other counseling 
services which may be based on the same or other philos- 
ophies of counseling Such an accumulation of data should 
lead to even moie specific desciiptions in which lecorded 
intei views would piobably piovide the raw data Ultimately, 
It should be possible to determine expeiimen tally which coun- 
seling procedures are most effective with what types of 
pioblems This is the objective of evaluative research in the 
field of counseling 
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A COMPOSITION TEST FOR FOREIGN 

LANGUAGES 


LAWRENCE ANDRUS 
University of Chicigo 

T his paper discusses a type of French composition 
test, developed at the University of Chicago, which has 
m practice yielded remarkably good results as a measuring 
instiument and has proved very stable, i c , has given com- 
paiable scoics fiomyeai to year 

The Lest was developed as a part of the comprehensive 
examination in French 104-105-106, the sequence in Interme- 
diate Flench given in the College of the University of Chi- 
cago The College, as the term is used at the University of 
Chicago, includes the yeajs corresponding to the fieshman 
and sophomore years of the tiaditional foui-year piogiam 
The prerequisites for admission to French 104-105-106 are 
two units of high school French oi the successful comple- 
tion of Fieiich 101-102-103, the sequence in Elementary 
French Students in the College, as contrasted with more 
advanced students who desire to offer Fiench 104-105-106 
as an elective in a held related to their major field, may gam 
credit for the sequence only by passing the comprehensive 
examination given at the end of the Spring Quaiter The 
great majority of students in the course aie College students 
Since these students pass oi fail solely on the basis of the 
compi ehensive examination, the staff of the couise and the 
examiner attempt, m every possible way, to make the exami- 
nation as valid, as reliable, and as disci iminating as they can 
In the attempt to seeme greater reliability, objective ques- 
tions, 01 questions which can be scored with high objectivity, 
have been devised 

The Announcement of the College for 19^0 41 describes 
Flench 104-105-106 thus “The primary objective of the sec- 
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(jnd-yciir sequence is the stanclaidizntion of the language abilh 
Lies To that end theic is contimunis tiammg m foimal and 
ijifoimal written and oial cvpiessioii, auial comprehension 
and the nccui.itc cleteimiiiiition of the value of the punted 
woid Appioxiniatcly twenty-five hunched pages aic read, 
with lepoits, following individual piogiams ” This statement 
iji a fan dcsciiption of the couise as given in the piecediiig four 
yeais, the pcuod coveied by this investigation The type of 
test hcic discussed was uitendcd to measuie the outcomes of 
tiaining in wiltten expicssion Tt was fust used experimentally 
in the comprehensive examination of June, 1934 ^ In siibstan 
tially its picsciit foimi it was nicluded as a pait of the 1933^ 
examination, and retained m the following years with mmoi 
changes In the physical picseiitation 

The essential fcaUucs of this type ol test aie as follows 
ii Kieiich passage is chosen which, in the judgment of the staH 
and the examinei, contains matcinil suitable foi testing at the 
level of the com sc, fiom the point of view of both vocabu- 
Iiiiy niul syntax It should be emphasized that the choice of 
an appropiiatc passage is cxticmcly impoitaut, if the test is 
to yield maximal icsiilts It may be ncccssaiy to leacl many 
pages hcfoie a suitable passage is located This passage is 
then translattcl into good Kiiglish The next step is to go 
thiough the Ficnch text and delete certain words and phrases 
The coriespondiiig paits of the English translation are iitidei- 
lined and numbered to agicc with the numbers replacing the 
omitted words and phrases in the French passage The stu- 
dent IS icqiiiied to complete the French passage in accordance 
with the English tiaiislation He is guided in this task by 
the numbeis and the underlining 

A sample taken from the June, 1939, examination will give 
,i better idea of the physical auangement of the test than a 
lengthy explanation 

^Sfcc Ern«8l lUcIcn nnd John M SlDlnaker, "A New Xype of Comprehen- 
sive Foreign Lnngunge Test,” Fhe Modem Language Journal, XIX, 2 (Novem 
bCT, m^), 81-92 
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TiausliiUon of hiPfic/i Test on Opiiositc Tag*. 

The old marquis de la T our-Samucl, (1) eiff/ity^iwa years old, aioae and 
came (2) to lean against the mantelpiece He said (3) in his (4) sotnevihat 
1 1 enibliuff voice 

*'{5) I, too, know a itrange thing, so strange (6) that it has been the obsca- 
sjon of my (ifc (7) It is now fifiy^six years (S) si7icc this adventure (9) hap- 
pened to me, and (30) a month doetn*t go hy (11) <zi’j;/joii/ my seeing it again 
fji a dream (32) There has remnwed to me from that day a mark, nn imprint 
of fear, do joii undcrstnnd me? Yes, (13) I vndeiweiit horrible fright, (14) 
for ten niinutes, (15) in such a way thnC since that hoiii (16) a kind of constant 
terroi (17) has lemained (IS) in my soul (19) Unexpected noises (20) make 
me start, (21) objects (22) that I make out (23) pootly m (24) the e<iiening 
shadow give me (25) a mad desire (26) to inn away Finally, I'm nfrnid (21) 
at night 


"Oh I (2S) I shouldtf'l ha^e admitted (29) that (30) before having arrived 
nt my (31) picsenl age Now 1 can say (32) everything It is perimited 
(33) not to be brave before imaginary dangers, when (34) yon are eighty two 
Before real dangeis, (35) I have ncvei letreated, ladies" 
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Certain v9ords and phrases have been oraiUed from the following French 
pjiaage, and a number lins been aubalitntcd for eicli omitted word or phrase 
In each numbered space nt (he nglu, write m FRENCH the -inpropnatc word 
or pbrnsc He sure that yuui Innslalion fits the French context An English 
iranslntion of the p issngc is given on the pofic opposite, the lianalation of each 
omitted wokI or plurtse is underlined and preceded by a number which cor- 
rcapoiuls to ilic iiiiniliei m the French passage for ench m icfcrence Note that 
dierc 19 not ihvays exact concHpondence m foim between the French and ihe 
English I he old mciiqnis uses the coiivenUliovdl, lhat is, uifoiml, style, until 
he begins to tell his story, sshirh is ui iilcujiy si\lc 
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A casual inspection of the sample suffices to reveal that this 
test foim makes possible the use of a gieat vaiiety of items, 
both as to content and as to length The items may all be 
cJassiiied iindei one heading usage^ with subclassification 
imdei active njocabnlai y (including idioms) rndgiamina) (in- 
cluding syntax) By the piopci choice of items tested, it is 
possible to vaiy at will both the level of difficulty and the pro- 
portion of vocabulaiy items and grammar items In this way, 
validity with leferencc to specific objectives and content of a 
given conise of study may be bmlt into the test For instance, 
in Fiench 104-105-106 at the University of Chicago, a com- 
mon piactice has been to lestiict items used m this test to the 
2,500 woids of highest fieqiiency in the Vander Beke Ftench 
JFoid Book^ A slmilai procedure may be followed with 
jespecC to idioms, by using the Cheydleui Fiench lAtovi List ^ 
Note that both upper and lowei limits may be adopted At 
piesent, giammai items must be validated on the basis of text 
books used and the subjective judgment of the uistiucting 
staff When the Fi cnch Synia\ Count, begun iindei the direc- 
tion of the late Professor Coleman, and now pioceedmg undci 
the direction of Professor Keniston, is finally available, theic 
will be an objective cntenoii of difficulty for Fiench syntac- 
tical consti actions In Spanish, this invaluable aid has already 
been published ^ 

Theoretically, the most disci iminating type of item for 
use m an achievement examination is one answered coirectly 
by 50 per cent of the gioup taking the examination ^ Iti 
practice, we almost nevei find a test containing even a majority 
of items of this type, except m the case of standaidized tests 
which have been refined by statistical pi ocediues, and even then, 

^Geoj'ge E Vander Bekc, French ff'ord Book (New Yoik Mncroillan, 
1929) 

^P'reclenc D Cheydieur, French Idiom iu/j Haied oti Running Count of 
J,WJ000 Hoards (New Yoik Macmillan, 1929) 

^Hayward Kcniaton, Sfamsh Sy7Ua\ List (New York Henry Hein & Co, 
1937) 

“Thelma Gwinn Thuratonc, “The Difficulty of n Teat and its Diagnostic 
Value,” The Journal of Educational Psjfchologyj XXIII, 5 (MiVi 1932), 335-343 
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pciliaps only with icfeience lo the gioiip on winch the test was 
standjiidizcd The classioom teachci inteiested in using the 
kind of test heicin discussed would obviously have neithei the 
time nor the statistical knowledge to go thiough the various 
steps lequiied to develop a test composed laigely of the most 
disciiminatiiig items, A fan appioxlination can, howevei, be at 
tamed by lemcinbeiing that items that will be passed by prac- 
tically all the group of students, oi by almost none, have veiy 
little value for disci imination They might be called "dead 
wood ’* The experienced teachei and his colleagues can, by 
subjective judgment, identify many such iinpiofitable items 
Repeated use of a given test foim and inspection of the le- 
sults (not ncccssaiily involving a foiiiial item analysis, al- 
though that IS always dcsiiable when piacticable), will tend 
to bring the Lcachei’s subjective judgment of the woith of an 
item closei to an objective evaluation It goes without saying 
that the value of an item as icgards disci imination vanes 
with the level of instiuction and the content and method of 
the com sc, and should always be estimated in tcims of these 
latter, To take a Jiypothetical example, in one school an 
Item involving a paiticulai use of the definite aiticle in Flench 
might be highly disci immating, Avlicieas in a school in a neigh- 
boimg town, using a dideicnt couise of study and <i diffeient 
method, the students might have leceived so much dull on 
this paiticulai point that an item involving it would be passed 
by practically every student, and, hence, be of veiy little value 
for disciimination 

If the passage chosen, although otheiwisc desiiable, is 
judged to lack an adequate mimbei of instances of a pnrticulai 
constiiiction consideied inipoitant by the instiucting staff, it is 
fiequeiitly possible to add items involving this consti action 
by making slight changes in the Ficnch passage It may not 
even be necessaiy to change the linglish tianslation at all, 
aftci such revision has taken place. The vocabulaiy items 
can be conti oiled in like mannei 

The scoiing of this test can be made veiy objective. At 
the time the test is consUucted, as complete a key as possible 
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IS piepared, piefeiably by the entiie staff of the course This 
key facilitates the woik of the scoiei, who must, however, 
know the language thoroughly The scoring cannot be en- 
ti Listed to clerks Whcnevei the scoiei meets a couect 
answci not included in the key, he adds it to the key Even 
with the necessity foi consideiation of such answers, the 
scoLiiig IS veiy lapid In syntactical items, mmoi eirois m 
spelling and mistakes m accents aie disicgaided, piovided 
that the student uses coiiectly the constiuction on which the 
Item hinges The scoimg thus becomes neaily as objective 
as that of a multiple-choice test 


TABLE 1 


COMPOSITION IFSl — 

■ l RPNCH 

104-105-106 



1937 

J93S 

1939 

1940 

No of items 

112 

100 

100 

100 

No of points 

No of points in comprehensive 

112 

100 

100 

100 

examination 

545 

495 

485 

485 

Mean 

5209 

5175 

45 96 

54 52 

Standard deviation 

1780 

15 95 

14 86 

13 98 

Reliability* 

Standard cnoi of mcasuiemcnt 

,94 

94 

92 

91 

((T„ = (7t VI — 

Correlation with entile compic- 

4 36 

3 91 

4 20 

419 

hensive examination 

92 

91 

88 

.90 

No of cases 

44 

60 

78 

52 


^Estimated by Kudei-Richaidson formula No 20, 


_ n / ~ npq ^ 

"““n-lV ) 

Table 1 shows the mam lesults of a statistical analysis of 
the different foims of this test used m the compiehensive ex- 
aminations m Flench 104-105-106 at the Univeisity of Chi- 
cago duung the foui-yeai period 1937-1940 inclusive 

We note that the mean score m all four yeais was in the 
geneial neighboihood of 50 pci cent of the possible number 
of points m the test This is equivalent to saying that the 
aveiage item was answered correctly by about 50 pei cent of 
the group taking the examination In 1940, a few items were 

^See G F Kuder nnd M W Richardson, "The Theory of the Estimation 
of Test Relinbility," Psychometriha, II, 3 (September, li>37), 151-60 
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puiposely included which seemed a pjion rather easy foi the 
gioup tested, in ordei to test the effectiveness of such a priori 
judgment. These items weie answcied correctly by most of 
the students, .ind aie leflected in the highei mean score for 
the 1940 test 

111 each of the foiii forms of the test, the standaid devia 
tioii was laige enough so that the students' scoics weie well 
spread out, theicby facilitating classification of the students 
in lank oidei of meiit The differences in size of the stand- 
ard deviation from one year to anothei arc no gieatei than 
differences often found in administeiing the same test to two 
different gioiips of students, one slightly moie homogeneous 
than the other The spicad of students’ scores on this test 
IS thus quite compaiablc fiom yeai to ycai 

For a test of 100 items, a icliability of 90 is commonly 
consideicd good The lowest icliability coefficient estimated 
foi the foiii-yeai pciiod was 91 for the 1940 foim, the 
highest, 94 foi the 1938 foim (this is relatively bettei than 
94 foi the 1937 foim, since the latter contained twelve more 
Items) 

In none of the foui years is the standard ciior of meas- 
iiiement as large as one-tenth of the mean score, and in none 
IS it as laige as one-third of the standaid deviation These 
values aie satisfactorily low They indicate that chance 
eiror in mcasuiement has been kept within reasonable limits 
Note that the difference between the highest and lowest 
standard cnois of measuiement here reported is only 45 To 
illustrate the meaning of this slight difference between the 
two extremes, let us assume that a student in 1937 and a 
student In 1938 each have a score of 40 00 The chances 
are two out of three that the true score of the 1937 student 
lies between 35.64 and 44 36, and that the true score of the 
1938 student lies between 36.09 and 43 91 

In only one year, 1939, docs the correlation of students’ 
scoics on the composition test with then scores on the entiie 

li/o/c It will be recognized thnt not all statisticians would agree 
as to the validity of tlua mtcrpielation 
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compiehcnsive examination fall below 90 The 1939 dis- 
tiibijtion includes one case (that of a student not tegisteted 
for the coiiise) which is so imlikc the other cases in the group 
that It lowers the correlation by at least 01, It is customary 
to legard the correlation of a sub-test with the entire exami- 
ftation as spuriously high, since the sub-test is being coiie- 
lated with a test of which it foims a part In each of the 
foul yeais, howeveij the composition test represents only 
about one-fifth of the total number of points in the entire 
comprehensive examination, and yet the con elation of this 
pait with the entiie compiehenaive examination remains uni- 
foimly high, although a good part of the remaining material 
in the compiehensive examination has changed m character 
from yeai to yeai This phenomenon, taken in connection 
with the 1 datively constant mean, standaid deviation, relia- 
bility, and standaid eiior of measuienient, leads to the con- 
clusion that under the conditions prevailing m French 104- 
105-106 at the Univeisity of Chicago this composition test 
lepreseiits a pai ticularly stable type of measuring instiumejit 
Eveiyone must agiee that the best method of testing 
Flench composition would be to requiie the student to wiite 
a flee composition in French, if such tests could be scoied 
reliably Unfortunately, reliable scoring is in practice very 
hard to obtain In the few instances wheie moderate success 
has been achieved, the process requires a gitat deal of time 
and involves essentially using the services of a jury of experts 
Most teachers would probably agree that, at least at the 
lower and intermediate levels, the two elements which would 
assume the greatest importance in their judgment of a free 
composition in French are active vocahtilaiy (including idi- 
oms) and gramnai (including syntax) This type of test is 
capable of measunng students' achievement in these two ele- 
ments reliably and objectively The wuter feels that at the 
lowei and mierviedmU levels it is wiser to use a test which 
can do this than to luii the risk of unreliable measurement 
which use of free composition entails 

The results of the statistical analysis shown in Table I 

363 



I'DllCAllONAl ANP 1‘SYCIlOIOGirAT MI-ASURhMLNl 


can be accepted unquestionably only for Flench 104405-106 
at the Univeisity of Chicago There is no giiaiautee, but 
tlieie is a stiong piesiimption, that .i test of tins kind, con 
stiucLed elsewheic with equal care, and with due attention to 
the objectives, content and method of the course of study, 
would yield equally favoiablc lesultb The technique could 
certainly he applied to Spanish, Italian, and Poituguese as 
Avell as to Ficnch, the sentence sLiuctuic of Geiman might 
pi event the technique fiom being as effective in that language 
as in the Romance languages 
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PARI iP 

SIDNEY W KORAN 

JLmployment Boaid, Pennsylvini'i Depnitment cf Public Assisttince 

The Test foi Gi aphoiype-Addressoffraph Opeiatois 

T he position of giaphotype-addiessograph opeiatoi 
occuis in each of the four legional financial offices of 
the DeparLmeni of Public Assistance Since these offices weie 
conveniently located about the State and weie equipped with 
a sufficient inimbei of graphotype and addressograph machines 
to jjisiiie lapid and efficient conduct of the performance test, 
each of the foiii was used as an examination centei and the 
examinees were peinmtted to appeal at the one of their choice 
To minimize difficulties likely to aiise because some exam- 
inees might be unfamiliar with the paiticuiai models on which 
the test was to be given, the notification foim sent to each 
examinee included the statement “The examination to which 
you have been assigned has been designed to test youi ability 
to operate the Class 6300 Giaphotype and the Class 2700 
Addressograph “ 

The test consisted of the following two parts and was set 
up and scored so that much the gi eater emphasis was placed 
on Pait I 

I Embossing names and addresses on Addiessograph 
plates With the Class 6300 Graphotype 
II Pi ill ting cards fiom the embossed plates with the 
Class 2700 Addiessograph 

Standai dilation of the proceduie was achieved by (1) 
having all tests administeied under the supervision of turned 

iPurt 1 of this nrliclc apiiear?d iii the July Issue of this Journal 
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[tidivicluEils, (2) rcquiung the c^amiiieis to repejit nil diicc- 
tions to the cxfiminee veihatim fiom the copy, (3) provid- 
ing eacli examinee with a set of instuictions setting forth 
the nature of the examination he was about to take and ex- 
actly what was expected of him, (4) using a stop watch foi 
all timing, and (5) caiefiil mechanical inspection of every 
machine at the beginning ol each period of testing 

About 10 minutes bcfoie he was assigned to a machine 
each candidate was given a copy of the Instructions to Exami 
nces (see Exhibit G) and told to lead them caiefully and 
to lefer to them as often as he wished, When his turn ai 
rived he was assigned to a Giaphotype, furnished with a file 
drawei containing 22 plates and 20 fiames, and given five 
minutes to familiari/.e himself with the machine and to praC’ 
tice embossing two of the plates At the cxpnation of the 
piacticc period the exaniiiier collected the two piactice plates, 
fuinished the examinee with the list of names and addresses 
to be embossed, and read aloud the appropriate sections of 
the Instuictions to Examinci (sec hAliibit H) At the expira- 
tion of 10 minutes the examinee was cliiccted to stop emboss 
mg and to place the completed plates into frames He was 
then assigned to an Addressograph and required to print a 
card from each embossed plate. If the examinee was unable 
to operate the Adcliessograph snflicieiitly well to punt a leg- 
ible copy from each plate, the examiner had an assistant print 
the plates on a strip of papci so that a record of the candi- 
date’s performance on the Graphotype would be available for 
scoring 

As with the Telephone Operator test (described in Pait 
I of this article), the scoring proceduic was designed to elimi- 
nate those whose performance fell below the standard estab- 
lished as the minimum acceptable, and to produce quantitative 
ratings rcllccting the lelativc operating .nbility of those who 
survived that elimination In establishing the qualifying 
point, consideration was given to (1) the lequriements of 
the job, (2) the calibre of the individuals employed and avail- 
able for employment, and (3) data on the agency’s previous 
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experiences xvith a perfoimance test for this type of position 

The piocediire followed m scoiing the plates of those 
who met this criteiion took into consideiation both the speed 
and the accuracy with which the plates had been embossed 
The examinee's law score on the Giaphotype portion of the 
test was detei mined by siibti acting his enoi acoie — computed 
by counting each deviation from the key® as one eiior — fiom 
the total number of strokes completed Examinees who dem- 
onstiated ability to opeiate the Addiessograph were given 
additional ciedit up to 10 pci cent of the total allowed for 
the complete perfoimance test Keys weie constructed which 
leduced the scoiing task to a loutuu operation 

The Test fof TabttiaUng Machine Opeiatois 

The position of tabulating machine opeiator (IBM equip- 
ment) occurs only m the operating agency's State office In 
administering the test in cities othei than Harrisburg it was 
theiefoic iiecessaiy to aiiange to use the facilities of the 
IBM Seivicc Bui can 

To discouiagc individuals whose entire practical experi- 
ence had been confined to the opeiation of sorters, numeric 
accounting machines, or Poweis equipment ft om reporting 
for the performance test, the following statement, intended as 
a rcmindei, was included in the notification form sent to all 
examinees "As pieviously indicated, the examination for this 
position has been designed to test your ability to operate both 
the IBM hoiizontal counting soiter and the IBM alphabetic 
.iccoiintmg machine ” 

The test consisted of a two-pait exercise In Part I the 
examinee was lequired to (1) wire the plugboard of the 
alphabetic accounting machine for listing and for totals, (2) 
use the horizontal sorter, and (3) adjust and operate the 
alphabetic .iccountjug machine so that ceitain alphabetic and 
numeric data fiom pieviously punched cards would be listed 
the same way as the data shown on the specimen form pro- 

SExamplea of deviationa penalized were (1) incorrect letter or number j 
(3) spacing eiroi of any kind, including littc space, (3) insertion of a letter 
or nuirber, and (4) omisaion of n letter or number 
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vided with the Instiuctions to Examinees (Exhibit I) p^it 
11 of the exeicisc was an extension of Pait I in that it lequired 
the examinee to list specific data from the tabulating caids 
aftei (1) wiring the same plugboard to piovide foi numeric 
coiitiol and subtotals, (2) ic-soiting the punched caids, and 
(3) making scveial additional adjustments on the accounting 
machine 

About iO minutes befoic the candidate was lequiied to 
start the actual test, he was piovided with a copy of the In- 
stiiictions to Examinees and told to lead them caiefully and 
to icfci to them as oltcn as necessaiy thioughout the exami- 
nation Attached to the sheet of Instiiictions weie a sample 
punched caid and a specimen sheet showing the foim in which 
the data weie to be listed by the alphabetic accounting ma- 
chine in Pait I of the cxciuse A sample caid and a poition 
of the specimen foim sheet aie icpioduced as Exhibits J and 
K lespectively 

As soon as the examinee was icady to start the test he was 
piovided with a plugboaid, an adequate supply of the vaii- 
ous sizes of wiics needed in making the connections, and, foi 
refciencc piuposes, a typc-bai layout and a plugboard dia- 
gum The examinee was then lemiiided that the time limit 
for the entile test was one hoiu and forty-five minutes 

Elapsed time was recouled by means of an electiic job 
clock by stamping the staiting time and finishing time of 
each opeiation on the examinee's job caid Since it was easiei 
to secure the use of pliigboaids than tabulating and account- 
ing machines, the equipment at each centei included about 
three times as many plugboaids as it did pieces of mechanical 
equipment. As a lesult of this it was sometimes necessaij 
foi an examinee to wait a few minutes foi his tiun at an alpha- 
betic accounting machine At no Lime, howevci, was it found 
that he was lequiied to wait loiigci than 10 minutes, and 
this "lost time" was, of coiiise, automatically taken care of 
by the job clock timing method employed The small delays 
caused by having seveul times as many examinees wiiing plug- 
boaids as could be accommodated at the machines weie so 
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slight as to cause vutually no inconvenience to the candidates 
On the othei handj the saving in time and expense which le- 
suited fiom following this pioceduie was consideiable, 

When the examinee had finished wiiing the plugboaid to 
his satisfaction, one of the examineis inspected it to make 
cel tain that no connections had been made which would be 
likely to damage the machine The examinee was then given 
A pack of 35 punched tabulating caids and diiected to con- 
tinue with Pait I of the test No limit was placed on the 
luimbei of sheets of papei the examinee may have found it 
iiecessaiy to use noi on the numbei of times he was peimitted 
to make changes in the plugboaid wiling oi machine adjust- 
ments He was Lold to wiite his identification numbei on 
each sheet and to write “final copy” on the one he wished to 
submit for scoiing When Pait I had been completed, the 
examinee continued immediately with Pait II 

Duiing the test the examineis made no attempt to latc the 
candidates on such points as the coriectness of their paiticu- 
lai appioacli, the acceptability of their work habits, noi, as 
alieady mentioned, the numbei of times they found it neces- 
sary to change the wiling oi leadjust the machine, The only 
factors taken into consideiation in scoiing the test were (1) 
the accuiacy with which the assignments had been carried out, 
as shown by the finished pioducts, and (2) the length of time 
consumed by the examinee in completing both parts of the 
exercise 

Reproduced as Exhibit L is a copy of the lating foim 
which has been maiked to show the scores of a typical exam- 
inee The number in the parentheses aftei each item on the 
latmg foim is the maximum score obtainable foi that item 
An examinee completing both paits of the exercise correctly 
within 45 minutes would receive 30 points for Part I, 30 
points foi Part II, and 40 points for finishing within the 
minimum time bracket, making a total score of 100 A func- 
tional bieakdown of the 60 points assigned to Parts I and 
II shows the following sorting, 10 points, location of alpha- 


369 



EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


betic and numcuc hekls, 30 points, totals and subtotals, 20 
points, 

The Test foi DtiplicaUng Machine Operators 
The position of duplicating machine opciatoi occurs m 
the State oflices of the opeiating agency and the merit system 
agency Pei sons filling these positions aie requiied to opei- 
ate several models of numeogiaph and miiltilith machines As 
most of the woik involves the contitiLious use of the Class 
1200 miiltilith and the Model 100 mimeograph m the pei- 
formance of a laige vaiiety of duplicating jobs, and persons 
who can satisfactoidy opciatc these models oidinanly have 
very little difliculty opeiating the oldei and Less complicated 
types of equipment, the peifoimance test was built aiound 
these paiticulai machines, and the examinees weie so informed 
well in advance of the date of the examination 

A copy of the Insti actions to Examinees (Exhibit M) 
was fuinishcd each candidate at least 10 mimites before he 
was reqimccl to begin the test He was told to lead these 
Irtstiuctions carefully and to keep them with him for refer' 
cnee throughout the examination 

The test foi mimeogiaph operatois was administeied first 
Each examinee was piovided with n mimeograph stencil into 
which a solid box of typcvviitten material measuiing 2Ji'* by 
4)4" had been fleshly cut, 75 4" by 6" white file caids, and 
75 sheets of Icttei-size mimeogiaph papei on which a 
by S}i" frame had been punted. The examinee was then 
refen ed to his punted instructions which directed him to (1) 
place the stencil on the cylinder, (2) adjust the machine and 
duplicate 25 cards so that the material which had been typed 
on the stencil was ccntcicd on each caul, (3) leadjust the 
machine and duplicate 25 sheets so that the typed material 
was centered within the piepimted fiame, and (4) remove 
the stencil and picpare it to be filed foi future use The 
frame pieprinted on the lettei-sl/e paper was located In a 
position which icquired maximum adjustment of the machine's 
margin guides before the typewritten matciial on the stencil 
could be made to punt within the required borders 
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When he was ready for the multilith poitioii of the test, 
the candidate was provided with a photographic multex plate 
containing typewjitteii mateiiaJ measiiiing 6" by 8", 75 sheets 
of lettei-size bond papei on which a by Sj/i" frame 

had been printed, and a supply of platex, keepeze, blankrola, 
lepelex, and absoibent cotton He was then lefei led to h/s 
printed instiiictions which directed him to (1) apply the pla- 
tex, (2) put the plate on the machine, (3) adjust the machine 
and duplicate 25 sheets so that the typewiitten mateiial on 
the plate was centered within the preprinted frame, (4) re- 
move the plate and prepare it to be filed for future use, and 
(S) clean the blanket Two forms of the multilith plate weie 
used alternately These forms differed fiom each other only 
in the location of the typewritten material on the plate and 
weie designed so that, while all candidates weie required to 
make exactly the same kind of adjustments to center the mate- 
rial properly, the exammcis were relieved of the necessity of 
setting up the machine after each run 

The examiners observed the candidates from a reasonable 
distance throughout the test in order to complete the rating 
form shown as Exhibit N Timing was accomplished with a 
stop watch, and considerable use was made of the lemaiks 
column of the lattiig sheet to lecoid all occur lences likely to 
contribute toward a fair evaluation of the examinee’s per- 
formance Whenever necessary, candidates were told how to 
turn on the particular model of equipment on which the test 
was given Certain other bits of information, such as the use 
of the ink rolls on the multilith and the side margin adjust- 
ments on both the multihth and the mimeograph were also 
given, but careful note was made of the circumstances in each 
case so that the stipulated penalties could later be subtracted 
The half hour time limit on each machine was mentioned 
111 the Instiuctions to Examinees but was not particularly em- 
phasised, As the candidate started each part of the test the 
examiner said, “You will be allowed up to 30 minutes to 
complete this part of the test The tune you actually con- 
sume will enter into the computation of your score, but you 
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ought not to woik so lapidly that the quality of your work 
siiffeis ” Veiy few candidates icquiiccl moic than 15 minutes 
to complete the mimeogiaph test oi nioie than 20 minutes 
for the multiUth test, and most of those lequuing moie than 
this time weie examinees who cithci got oh to a bad stait or 
weic obviously so iinfamiliai with the equipment that they 
wcic simply pciseveimg while hoping foi a miiade to occur 
Pei sons in the lattci gioup weic encouiagcd to continue as 
long as they did not endaiigei the equipment The additional 
time lequiicd to do this was chcei fully chaiged to public 
relations when it was discovciecl that, instead of lationalizmg 
that if they had been given moie time they would have suc- 
ceeded, these candidates eventually insisted on withdrawing 
of their own accoul and almost invaiiably thanked the 
examiners foi "being so patient with me and giving me 
every bicak " 

Scoring was acuiiuphshcd by determining the number of 
points earned by the examinee foi coircctly accomplishing 
each of the items listed in the schedule of ciechts (Exhibit 
0), and then entering the appiopiiatc amounts in the spaces 
provided on the summaiy sheet (pAhibit P) As finally 
worked out, the schedule of cieclits piovided a weight of 60 
foi the rnultilith poition of the test, and 40 foi the mimeo- 
giaph 

In establishing the numbci oi ci edits to be allowed foi the 
successful accomplishment of each "item" of the test, con- 
sideiation was given to the lelative difficulty of the paiticulai 
function undci consideiation Thus, m scoung the mimeograph 
poition of the test, twice as much cicdit (8 points) was 
gi anted when the examinee’s finished pioduct presented evi- 
dence of collect sidc-maigm adjustments as when the vcitical 
maigins wcie satisfactoiy (4 points) Foi the miiltihth, on 
the othei hand, moic cicdit (10 points) was given for piopei 
adjustment of the side maigins llian foi having the veitical 
margins collect (6 points) 
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Exhibit G 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
OF THE 

DEPARTMENT OF PUBLIC ASSISTANCE 

Harrisburg 

PruroRMANCF Tfst — Series 1900 
GRAFtioTYPr AND Addrussqgraph Machinc Operators 

July 1940 

INSl RUCTIONS TO EXAMINEES 
Important Failure to follow instructions may 
result in disqualification fioin the evamination 

Study these instructions carefully When i^ou aie ready to begin 
the examination, signal the Examiner He will assign vou to a machine 
and furnish you with the mateiial with which you are to work 
Graphotype Machine 

The examination for this machine consists of embossing a number 
of names and addresses in accordance with the form shown in the 
attached sample 

As soon as you have been assigned to a machine, the Examiner will 
furnish you with a file drawei containing 22 plates anti 20 frames. You 
Will be given 5 minutes to familiarize youiself witii the maclime during 
which time j'oii maj use 2, and only 2, of the plates to practice 
embossing. 

At the conclusion of the practice period the Examiner will collect 
the 2 practice plates and give you a mimeogiaphcd list of names and 
addresses winch you aie to begin embossing as soon as he gives the signal 
to "Start " 

Continue embossing until the Examinei calls "Tiiuc ” Do not put 
the plates in the frames as they are embossed, 3'au will be required to 
do that later 

"Time" will be called at the end of exactly 10 minutes, 

The list of names to be embossed has purposely been made longer 
than even the fastest opeiators are likcb^ to be able to complete If the 
Examiner calls "Time" while vou are in the midst of embossing a plate, 
you may remove the unfinished plate from the machine, but you must 
not continue to emboss if 
Inseitiriff Plates in Fj nines 

As soon as the Examinei tells you to do so, place each embossed 
plate m the lower part of a frame and arrange all frames in the file 
drawer so that they will be ready to lun through the Addressograph 

When you have finished this task, the Examiner will provide you 
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with an envelope contfiiJunE twenty 4" by 6" cards (on which your 
Identification Number has been punted) and assign you to an Addresso- 
graph 

Addressograph Machine 

Tlie cvammatioii for this machine consists of printing from each 
plate that you have embosbcti 

The machine will be set to pimt consecutively, and you will be 
required to make A single impicssion on each 4'' by caid m the 
position shown on the attached sample, 

PLACE YOUR APPOINTMENT SLIP, THE INSPECTION 
SHEET, AND THE TWENTY CARDS INTO THE 
LARGE ENVELOPE AND SEAL IT 
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Exhibit H 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
OF THE 

DEPARTMENT OF PUBLIC ASSISTANCE 

Harrisburg 

PErtPORMANCE Test 

Graphotypi- and Addeessograpu Machine Operators 

Series 1900 
July 1940 

INSTRUCTIONS 10 EXAMINER 

1 Read the INSTRUCTIONS TO EXAMINEES and become 
entirely familiar with their contents before attempting to administer 
the examination 

2 Examinees will be scheduled at tlic rate of tJirce or four per houi 
where one set of machines is available, and at the rate of six oi 
eight per hour where two sets arc available 

3 Do not admit anyone without an Admittance Slip unless he can 
establisli his identity as an examinee who has qualified for the 
machine test 

4 Provision sliould be made for tJie examinee to be comfortably seated 
away from the scene of the examination while he is awaiting his 
turn to operate the machines, 

5 About 10 minutes before the examinee is assigned to a machine, 
talcc his /rngeipiint, hand him a copy of the mimeographed IN- 
STRUCTIONS, and tell him to read them caiefuUy If he asks 
any questions, you may answei them, but it should not be necessary 
to furnish anv information beyond that already appearing in the 
INSTRUCTIONS 

6 When the examinee is ready to begin the examination and a 
Grapliotvpe machine becomes available, assign him to the machine 
and furnish him with a file drawer containing 22 plates and 20 
frames (all in perfect condition), 

7 Xell the examinee he may have five minutes to familiarize himself 
With the machine and may practice embossing on two of the plates 

S At the expiration of five minutes (or befoie, if tJie examinee says 
he IS ready to begin) collect the two practice plates and hand him 
the list of names to be embossed Then say 

*'DO NOT BEGIN UNTIL I GIVE THE SIGNAL 
EMBOSS EACH NAME AND ADDRESS ON A SEP- 

375 



EDUCAIIONAL AND PSYCHOLOGICAL MLA&URCMENT 

ARATE PLATE USING THE SAME FORM AS IN 
THE SAMPLE ATTACHED TO YOUR INSTRUC- 
TIONS COPY THE NAMES AND ADDRESSES 
EXACTLY AS THEY APPEAR, OMITTING ALL 
PUNCTUATION AT THE END OF 10 MINUTES. 
WHEN I CALL TIME/ YOU MUST STOP EMI30SS- 
ING ” 

9 Say to the c\ammc‘c< ^'VLEADY, START 

10 Permit the examinee to continue c\actly 10 tiimuies. Then say, 

“TIME STOP EMBOSSING " 

11 If a plate IS in the machine, peimit him to remove it Take away 
tlic list of names and all blank phtes Then say 

“PLACE EACH EMBOSSED AND PARTIALLY EM- 
BOSSED PLATE INTO TPIE LOWER PART OF A 
FRAME AND ARRANGE THE FRAMES IN THE 
FILE DRAWER SO THAT THEY CAN BE RUN 
THROUGH THE ADDRESSOGRAPII “ 

12 When the examinee has placed each plate in a fiamc, take away 
the unused frames, hand him liis envelope contammK the cards, and 
assiRFi him to an Addrcssosiapb Then sav 

“PRINT THE CARDS FROM THE PLATES MAKE 
ONLY ONE IMPRESSION ON A CARD AND IN 
APPROXIMATELY THE SAME POSITION AS 
SHOWN ON THE SAMPI/E ATTACHED TO YOUR 
INSTRUCTIONS THE ADDRESSOGR APH HAS 
BEEN SET TO PRINT CONSECUTIVELY GO 
AHEAD “ 

n When the examinee has made an impicssion from eacli plate, tell 
him to place the 20 cauls (punted and imprinted) Into the envelope 
together with his Adiaittance Slip and Instuiciions and seal the 
envelope 

Note If ilic examinee is unable lo operjic ilic Addressograpli sufficiently well 

10 print a legible copy from each phte he has emboased, have the phtes printed 

on n strip of paper so that a record of his peiformnncc on the Ornphotype will 

be available for acoring 

14 If at any stage of the machine opciation you and the Addicsso 
graph lepresentativE ,ue convinced that the examinee docs not 
possess sufficient knowledge of the operation of either machine to 
continue with safety to himself and without damage to the equip- 
ment, the test may be halted If tins betomes iiccessaiy, a full state- 
ment of the cncumstanccs must be written on the back of the 
Admittance Slip and signed by both j'ou and the icpresentativc 
In any instance in xvliich the plates themselves will be valuable as 
possible exhibits, they should be enclosed in the ex.imuiee's envelope 
before sealing 
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Exhibit I 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
OF THE 

DEPARTMENT OF PUBLIC ASSISTANCE 

Harrisburg 

PLB.roRfvfAN’cE Tesi — Serils 2100 
Senior Tabulating Machinp OpERAroR 
Decembei 1940 
instructions to examinees 
Important Failuie to follow instructions may 
result in disqualification from the examination 

Study these instructions carefully As soon as a machine becomes 
available, the Examiner will give you furthci instructions and furnish 
you with all necessary material 

TJic peifoimance test for this position will consist of a twopait 
exercise designed to determine your ability to opeiate the IBM Hon- 
7 ontal Counting Soi tei and IBM Alphabetic Accounting Mnchine The 
time limit fot the entire test is 1 hour and 45 minutes 

You will be given 35 tabulating cards (into which various data 
have been punched) and a plugboaid foi the Alphabetic Accounting 
Machine Note that the model ot the Accounting Machine being used 
has 32 counters and 55 type bars of which numbers 19 to 43 are 
alphflhetrc, 

The following operations should he earned out in the order indi 
Gated 

PART I 

1 Wire the plugboard so that the machine will list the following 
information exactly as slioion on the attached sainple 

Card Number 

Name (last, first initial, middle initial) 

Social Security Number 
Total Benefits Paid 
Weekly Benefit Amount 
Reason 

Note In addition to listing the data> the machine is to be 
wired to show tot*ils at the end of each of the following fields 
Total Benefits Paid (allow for six digits) 

Weekly Benefit Amount (allow for six digits) 

2 Sort the cards in "Card Number” order. 

3 Write your Identification Number in the space provided on 
Form I and list the data from the cauls to conform to the 
sample and the above instructions 
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PART II 

1 Wire the plugboiird to contiol on "Reason 

2 Sort the caids by "Reason,” diMcgardmg "Card Number " 

3 Set automatiL h.iiiimcrloek control to eliminate the listing of 
names, 

4 On .mother copy of Foim I write yom Identification Number 
in the space piovicicd and list the data fiom the cards, single 
spaced, to show Mibtot.ils for each reason (foi Total Benefits 
Paid and Weekly Benefit Amavmt), 

PLACE YOUR ADMITTANCE SLIP, THIS INSTRUCTION 
SHEET, THE PUNCHED CARDS, AND BOTH COPIES 
OF FORM I IN THE MANILA ENVELOPE AND SEAL IT 
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Exhibit L 

PnnroRArANcr Trsr — ^Tadulaiing Machinf Opfrators 

Siitirs 2100 


File Number 


D Z43J6 

Ailcghetiy 

Ident Number 

I cfrnl Cniinly 


IFtihilreio f}oiii E\n)intiaiion (0) 

TART I 

Snilin/^ by CniH Nitinbei {‘>) 

Locattan af Fields (1^) 

Caitl Number (1) 

Name (2) 

Social Scciiiity Numbci (2) 

Total benefits Paul (2) 

Weekly Benefit Amount (2) 

Reason (2) 

Totals 

Benefits Paid Column (2) 
Benefit Amount Column (2) 
^Irciiiacy of Totals (JO) 

Benefits Paid Column (5) 

Benefit Amtiunt Column (5) 

PART n 

liofifiiff by Reason (^) 

Location of hteids (1‘i) 

Card Numbci (1) 

Name ehnnnated (2) 

Social Sccuiity Numbci (2) 

Total Benefits Paid (2) 

Weekly Benefit Amount (2) 

Reason (2) 

Subtotals 

Beiiefitb Paid Column (2) 
Benefit Amount Column (2) 
Accutacy of Subtotals (10) 

Benefits Paid Column (5) 

Benefit Amount Column (5) 

Time (40) 

45 minutes or less (40) 

46 to 60 minutes (30) “18 mm 

61 to 75 minutes (20) 

76 to 90 minutes (10) 

91 to 105 minutes (0) 


Scored by 

(Form EB-742) 


SWK 


TOTAL RAW SCORE 
Cheeked by 
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Cxbbit M 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
OF THE 

DEPARTMENT OF PUBLIC ASSISTANCE 

Han isbiirg 
Perfoumanci Tesi 
Duplicating Machine Operators 
Series 2800 
November 1940 

INSTRUCTIONS TO EXAMINEES 

IvipQitant Failure to follow instructions may 
lesult in disquali/icatioii fiom the ejiaminatian 

Study these mstiuctions carefully As soon as a machine becomes 
available, the Examiner will give you further instructions and furnish 
you with all necessary material 

Mimeograph Machine Time limit, 30 minutes The Examiner 
will furnish you with ilic following material 
1 newly cut mimeograph stcncil 
75 4" by 6" cards 

75 sheets of pre-prmted by \ V' mimeogiaph paper 

(sample attached) 

The examination for this machine will consist of (1) putting the 
stencil on the cylmdei, (2) adjusting the machine and duplicating 25 
cards so that the matciial which has been typed on the stencil is cen- 
tered on each caid, (3) readjusting the machine and duplicating 25 
sheets so that the typed matcnal is centered within the pre-pnnted box, 
and (4) removing the stencil and piepanng it to be filed for futinc 
use (You may use as many cards and sheets of paper as necessary in 
setting up the machine, but do not waste any All matcnal will be 
considered in determining your score iii tlie examination ) 

WHEN YOU HAVE COMPLETED THIS PORTION 
OF THE TEST, PLACE THE STENCIL AND ALL 
VSEV CARDS AND PAPER INTO THE LARGE 
MANILA ENVELOPE 

Muhthih Machine Tune limit, 30 minutes The Examiner will 
furnish you with the following material 
1 photogiaphlc multex plate 

75 sheets of pre-pnnted by 11^^ bond papei (sample 

attached) 

Supply of Platex, Keepeze, Blankrola, Repelex, and absorbent 
cotton 
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The cxamJiintion foi tins machine will consist of (1) applying 
Platcx, (2) puttniR the plate on the machine, (3) adjusting the machine 
niid diiphcatJiiR 25 sheets so that the typed matciial is centered within 
the prc-pimtcd bo\, (4) icmovitii? tlic plate and preparing it to be filed 
for future use, and (5) Llc.iiiiiig the blanket (You may use as many 
cards and sheets of papei as necessary in setting up the machine, but do 
not waste any All matciial will he consideicd in deteimiiung your score 
in the examination ) 

WII^N YOU HAVE COMPLETED THIS PORTION 
OF THE TEST, PLACE YOUR ADMITTANCE SLIP, 
THIS INSTRUCTION SHEET, AND ALL USED 
PAPER INTO THE LARGE MANILA ENVELOPE 
AND SEAL IT. 
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Exhibit N 

COMMONWEALTH OF PENNSYLVANIA 
EMPLOYMENT BOARD 
OF THE 

DEPARTMENT OF PUBLIC ASSISTANCE 

Ilarnsburg 

Performangi< Tist for Dupucatino Machinf Operators 

Series 2800 


November 1940 


examiner's rating SIIEtr 
Mifneograpfi Machne 


Examinee's Id No 


OPERATION 

Card 

Paper 

Remarks 

Place stencil on cylinder 


X 


Adjust piper feed i 




Wnifc side nnrKiii adjustment 




Make vcitical margin adjustment 


\ 


Use of print recorder 


1 


Run copies 


i 


like olf stencil 

X 

1 


Clean stened 

X 

1 




1 


TIME START 

STOP 1 ELAPSED 


MiiUtltlh Machine 


OPERATION 

1 

PapCi 

Remarks 

Flatcx plaie 



Put on plate 



( 



Puil proof 



Locate foim in proper position 



Clean blanket 



Set counter ] 



Run copies 1 



Clean plate ' 



Tnkft olf plate 



Kcepeze 



Clean blanket 






TIME START 

] STOP RLAPSED 


Date. 


Kxaminer. 

Examiner. 

'183 
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Exhibit O 

PROCEDURE FOR SCORING DUPLICATING MACHINE 
OPERATOR PERFORMANCE TEST 

SERIES 2800 

sciirDuiL or cridiis 

Note' The c\amiiicc's latiiig foi cacli item is to be 
tlctcnniiicd in nccoi dance with ihc follow iii|; schedule 
and cntcicd in tlie 'ippiopiiate space on the sconiiE 
sheet (Foim EB-760) All latnips and totals must 
be chcckrd by a second scoicr 

Mwieotjiaph (40) 

Time (16) 

1 to 10 miiuitcs — 16 points 
11 to 15 111 unites — 12 points 
16 to 20 mimites — 8 points 
21 to 25 minutes — 4 points 
26 to 30 minutes — no cicdiL 

Stencil (2) 

Removed and cleaned piopcily (ticdit if checked on inting sheet) 
— 2 points 

4" hy cauls (11) 

Piacticc caids (not more than 15) — 2 points 
Number of copies (25 to 30) — 2 points 
Use of countci — 1 point 

Vcitical margin adjustment (80% of final copies with at least 
maiEin top and bottom) — 2 points 
Side maiRins (80% of final copies with at least margin each 
side) — 4 points 

iV/ 2 " by 11" pape, (11) 

Piactice sheets (not inoic than 15) — 2 points 
Numbci of sheets (25 to 30) — 2 points 
Use of countci — - 1 point 

Vertical maignis (80% of final copies not touching horiaoiital 
lines) — 2 points 

Side maignib (80% of final copies not touching vertical lines) — 4 
points 

Pemlty (-4) 

No 1 — If examinee was given information on side margin adjust- 
ment— iiii /iff c/ 4 points (but only when examinee has icceived 
credit for coircct side margin adjustment) 

Muhilttb (60) 
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Tu/te (24) 

1 to 10 iniJiutes — 24 points 
11 to 15 minutes — 18 points 
16 to 20 iniiuites — 12 points 
21 to 25 minutes — 6 points 
26 to 30 minutes — no credit 
Gleflti i>late (ciedit if checked on iatiii(i sheet) — 5 points 
Clean hlankiii (cicdit onlv if scconc? Iistine is checked on uting sheet) 
— 2 points 
h paper (29) 

Practice sheets (not more than 15) — 5 points 
Number of copies (25 to 28) — 3 points 
Use of countei — 2 points 

Side margins (at least 3/16" on each side) — 6 points 
Vertical margins (at least 3/16" top and bottom) — 10 points 
h\kmg (3) 

Evenness — 2 points 
Blackness 1 point 
Pejurllies (-10) 

No 2 — ^If examinee was given information on use of ink rolls — 

suhlract 5 pQinis 

No 3 — If examinee was given information on side margin adjust- 
ment — subtract 3 patnls (but only when examinee lias re- 
ceived credit foi coirect side margin adjustment) 

No 4 — If examinee left dii ink roll in contact noth plate cylindci — 
subtract 2 points 


385 



KDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 

Exhibit P 

PERFORMANCE TEST— DUPLICATING MACHINE 
OPERATORS— SERIES 2800 

County 

Operation of Mimeogi aph (40) 

Withdrew or was stopped by Examiners (F) . 

Time (16) 

Stencil (2) 

4" by 6^' Cards (11) 

Practice caicls (2) 

Number of copies (3) 

Veitical margin adjustment (2) 

Side margins (4) 

Sy^" by 11" Papa (11) 

Piactice sheets (2) 

Number of sheets (3) 

Vertical margin adjustment (2) 

Side margins (4) 

Operation of Miiltihth (60) 

Withdrew or was stopped by Exanipiers (F) , 

Time (24) ' 

Clean Plate (5) 

Clean Blanket (2) 

Sy," by 11" Paper (29) 

Practice sheets (5) 

Number of copies (5) 

Vertical margin adjustment (10) . . 

Side margins (6) 

Inking (3) 

Total 

Penalties: No, 1 No, 2 No. 3 No. 4 Subtract 

Total Raw Score 

Scored by^. Checked by — 

(Form EB-760) 


Itlent. Number 


File Number 
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THE VALUE OF INTELLIGENCE QUOTIENTS 
OBTAINED IN SECONDARY SCHOOL 
FOR PREDICTING COLLEGE SCHOLARSHIP 

L, D. HARTSON 
and 

A. J. SPROW 
Oberlin Coliege 

I N SELECTIVE admission to college, and particularly in 
the award of scholarships, it is the practice to request a 
report of scores made by the candidates in Intelligence tests. 
This study reports (1) the relative value of these different 
tests for predicting (a) total high school scholarship,^ (b) 
college freshman scholarship, and (c) seven-semester college 
scholarship; (2) the comparative validity of these tests and 
the Ohio State University Psychological Examination; (3) the 
average I.Q, of the student body, as determined by these 
various instruments; (4) comparison of the I.Q.’s of the 
Oberlin group with those of the Ternian-Merrill standardiza- 
tion group; (5) average freshman scholarship for students of 
the different I.Q. levels. 

A total of 835 freshmen entered the College of Arts and 
Sciences, Oberlin College, during the period, 1934 to 1940, 
for whom I.Q.’s were available, which could be identified with 
specific forms of tests, in groups large enough to warrant 
statistical treatment In six cases there were two scores, making 
the total of 841 in Table 1. Of these, 253 had progressed as 
far as their eighth semester. For these, the computations are 
based upon the scholarship record for seven semesters (those 

Vphe figures used in the computations of high school scholarship represent, 
not the actual grades, but "credit points” obtained by a system used to equate 
different grading schemes. 
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VALUE OF I.Q.’S FOR PREDICTING COLLEGE SCHOLARSHIP 

used for the award of Phi Beta Kappa). In order to make 
the scholastic records equivalent for the several classes of 
varying size, scholarship is handled in terms of proportional 
class rank. Because the reports did not, in all cases, specify 
the particular variety of Otis test employed, all the Otis scores 
have been grouped. All the I Q.’s here considered were de- 
rived from group tests. 

Res nils 

Table 1 repoits the coefficients of correlations (1) between 
the I. Q.’s on the Otis, Terman, Henmon-Nelson, National, 
and Kuhlmann-Anderson Tests, and scores on the Ohio State 
University Psychological Examination administered before 
matriculation in college as one variable, and high school 
scholarship; (2) between the above-named tests and first 
semester college scholarship; (3) between I. Q.’s on one or 
another of the first five tests and scores on the OSU test, 
administered during freshman week, with the means and 
sigmas. In the case of the National and of the Kuhlmann- 
Anderson tests the N is rather small, and all the data, there- 
fore, are less reliable than those obtained with the other tests. 
To obtain a basis for comparing the validities of the OSU test 
and each of the others, Table 2 reports, for each of the test 

TABLE 2 

CORRELATIONS BETWEEN THE OSU TEST SCORES AND SCHOLARSHIP OF 
THE GROUPS TESTED WITH THE OTIS, TERMAN, HENMON-NELSON, 
NATIONAL, KUHLMANN-ANDERSON AND OSU TEST (PRE-ENTRANCE 


group) with means and sigmas 


Test Group 

N 

Scholarship 

High Sch Freshman 

Means 


Otis 

444 

394 

579 

49.35 

28.95 

Terman 

221 

.337 

.550 

51.47 

28.00 

Henmon-Nelson . 

no 

.458 

.604 

48.59 

29.22 

National 

38 

.473 

.631 

54.18 

25.36 

Kuhlmann- 

Anderson 

28 

.633 

.564 

51.93 

26.63 

OSU Test 
(pre-entrance) 

258 

.510 

.629 

48.72 

28.72 
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groups, the correlation between scores in the OSU test and 
(a) high school scholarship and (b) first semester college 
scholarship, with means and sigmas. 

1. Relative Fall lit tics of the I.Q. Tests. A comparison 
of the validities of the different tests yielding I.Q.’s indicates 
that the Hcnmon-Nelso’n test gets first place. However, the 
relative deviate of the difference (k) between the correlation 
of Henmon-Nelson I.Q.’s and college scholarship (.480) and 
the corresponding coefficient for the Otis test (.364) is 
but 1.20. 

2. The Prediction of High School and College Scholar- 
ship. The coefficients indicating the relationship between I.Q, 
and high school scholarship range between .212 and .396. 
With the exception of the Kuhlmann-Andcrson test (for 
which there are but 28 cases) the I.Q.’s constitute a better 
basis for predicting college scholarship than they do high 
school grades When validated against college scholarship, 
the coefficients range between .287 (omitting Kuhlraann- 
Andeison) and .480 Ryrns and Henmon, who used I.Q.’s 
obtained from National Intelligence Tests, administered in the 
4th to 8th grades, also found a closer correlation between 
I.Q. and first semester college scholarship (.“154) than be- 
tween I.Q. and total high school scholarship (.426) (2) 
Higher validities for the college criterion were also obtained 
for both of the OSU examinations. With the pre-entiance 
OSU Test the coefficients arc .365 and .474, and with the 
Fieshman Week test, they arc .510 and .629, for high school 
grades and college scholarship, respectively. These results 
substantiate previous findings at Oberlin. For the 511 men 
and 609 women who entered as freshmen during the period, 
1931 to 1934, the correlation between college scholarship and 
OSU Test intelligence is represented by coefficients of .605 and 
.574, for the men and the women, respectively, whereas the 
correlation between test intelligence and high school scholar- 
ship is represented by coefficients of .398 and .380 (3). It 
will be noted that the OSU Test scores have higher validity 
than does the I.Q., as indicated by the coefficients obtained 
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TABLE 3 

COMPARATIVE VALIDITIES OF THE OSU TEST AND THE I. Q. TESTS FOR 
PREDICTING HIGH SCHOOL AND COLLEGE SCHOLARSHIP 


Test Gioup High School Scholai ship College Scholarship 



OSU Test 

I.Q. 

OSU Test 

I.Q 

Otis 

. .394 

.322 

.579 

.364 

Terman 

. .337 

.281 

.550 

.403 

Henmon-Nelson - , - , 

. 458 

396 

.604 

.480 

National 

.473 

.212 

.631 

.287 

Kuhlmann-Aiiderson 

.633 

.247 

.564 

.178 


when correlations were computed for each of the I.Q. popula- 
tions between the scores made with the OSU Test and the two 
scholarship criteria (Table 3) 

The OSU Test is designedly a more difficult one than the 
other tests. Although some tests have as many items as the 
OSU Test, none requires as much time. All of the I.Q. tests 
are time limited, maximum time being 30 minutes, whereas 
the OSU Test was administered by work-limit method, stu- 
dents usually taking at least two hours. 

3 Comparative Validity of Pre-entrance and Freshman 
Week OSU Test Scoies. That the higher coefficients obtained 
for the OSU Test may not be due entirely to its greater 
difficulty, however, is suggested by a comparison of the co- 
efficients obtained for the OSU Test under two sets of condi- 
tions. There were 258 students who took the OSU Test some 
time before entering college who were re-examined with this 
test during their Freshman Week. (In some instances the 
same form of the test was used, but usually it was another 
form ) The Freshman Week test yielded substantially higher 
validity figures with both criteria . .5 1 0 as compared with .365 
for high school scholarship, and .629 as compared with 474 
for college freshman scholarship These coefficients, with the 
means and sigmas, are reported in Tables 1 and 2. It will 
be noted that the group given the OSU (pre-entrance) Test 
had distinctly lower scholastic records than the others and 
that they also displayed greater variability. This Is to be 
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explained by the fact that the great bulk of these students 
were given the test because their high school record made their 
admission questionable, On the other hand, the group con- 
tained some who, being of exceptionally high caliber, were 
applying for scholarships. In most cases, it may be presumed, 
the OSU Test was administered under conditions of greater 
motivation than prevailed with the I.Q. tests. 

4. hiic7 CO) relations Between the Test Scores. The inter- 
con elations between the scores made on the OSU Test and the 
different forms of I.Q. range from .456 to .610; the sequential 
order from the higher to lower coefficients being; Terman, 
Henmon-Nelson, Otis, National, and Kuhlmann-Anderson. 

5. Validation of Tests jlgatnst Total College Scholarslnp. 
There are 253 students for whom scholastic grades are avail- 
able for seven semesters of the college course. Because the 
numbers were too small to warrant separate computations for 
each of the tests, all of the I.Q.’s were combined and the 
validity coefficients computed, using both the one- and the 
seven-semester criterion. Validity coefficients were obtained 
for the OSU Test for the same population. These are given 
in Table 4. The two validity figures for the I.Q.’s are .341 
and .319, and for the OSU Test scores the figures are .501 
and .438. Although scores on the OSU Test are more valid 
bases for prognosing college grades than is the I.Q. at both 
levels, their superiority for predicting total college scholarship 
is less than when used for predicting freshman grades. From 
Table 4, one may also note that, as in the computations re- 

TABLE 4 

COIUIELATIONS BETWEEN I. Q.’s AND OSU TEST SCORES AND (1) HIGH 
SCHOOL SCHOLARSHIP, AND SCHOLARSHIP FOR (2) ONE AND (3) SEVEN 
SEMESTERS ; WITH MEANS AND SIGMAS 


Test Score N High Sch. 1 Sem. 7 Sems. Mean Sigma 

I. Q 77. . . 253 l7l ^341 ^^9 12124 10^ 

OSU Test 253 .307 .501 .438 50.95 29.85 

Mean 75.82 53.37 47.16 

Sigma 17.19 25.68 29.00 
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ported for the other populations, the I.Q. (and OSU Test 
score) shows a closer relationship for college freshman 
scholarship than for high school scholarship 

6. Aveiage I.Q, of the Oberltn Student Body. The mean 
I.Q. of the 835 freshmen, as measured by these group tests, 
is 121 06. There is substantial agreement on this point be- 
tween the Otis, Terman, and Henmon-Nelson tests (see 
Table 1 ) . The cases measured by the National and Kuhlmann- 
Anderson tests, for which the means are 127.21 and 124.96 
respectively, are too few to influence the general average 
materially. The average for the group of 253 who attained 
senior status is 121.24, thus indicating that practically no 
selection occurred between the freshman and the senior year 
in terms of I.Q. This is corroborated by the OSU Test stand- 
ing of the freshman and senior groups. In terms of local 
freshman norms, the mean score of the 835 students is 49.97, 
thus indicating that they are an almost completely perfect 
sample of the Oberlin first-year population. The mean score 
of the 253 who became seniors is 50.95. The seniors do, how- 
ever, constitute a somewhat selected group in terms of college 
scholarship. This is indicated by the fact that, whereas the 
mean freshman scholarship of the entire group is represented 
by a proportional rank of 49.55, the mean freshman rating of 
those who persisted until they reached senior status is 53.37 — 
the larger figure represents higher scholarship status — and 
the mean scholarship of the 588 who had not become seniors 
is 47.91. The critical ratio of the difference between the fresh- 
man scholarship of those who became seniors and those who 
did not is 2.73. 

7. Comparison of Oherhn Students with the Terman- 
Merrill Standardization Group. Figure 1 presents a graphic 
comparison of the Oberlin students with the normal group of 
2904 used in the standardization of the Terman-Merrlll Binet 
test (4, p. 37). The numbers and proportions of the Oberlin 
population at the different I.Q. levels are reported in Table 5. 
The I.Q.’s of the Oberlin group range from 92 to 169, 99 per 
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TABLE 5 


FRESHMAN AND SENIOR SCHOLARSHIP OF STUDENTS OF 
DIFFERENT I. Q. LEVELS 


I Q. 

N 

Entrants 

Scholarship 
Mean Range 

N pos- 
sible 

N ac- 
tual 

Seniors 

% 

Scholarship 
Mean Range 

166-170 

3 

0.4 

97 7 

97-99 

3 

2 

66.7 

96,0 

93-99 

161-165 

0 




0 





156-160 

0 




0 





15WS5 

1 

O.l 

94*0 

94 

1 

1 

100.0 

87 0 

87 

146-150 

13 

1 6 

72 9 

13-95 

5 

3 

60.0 

63,3 

30-90 

141-145 

14 

1 7 

72.1 

26-97 

7 

5 

71 4 

66.0 

12-84 

136-140 

44 

5 3 

70 9 

5-98 

16 

13 

81.3 

51.2 

1-99 

131-135 

57 

6.S 

59.8 

4-99 

IS 

14 

77 8 

49.6 

13-99 

126-130 

110 

13.2 

58.4 

1-99 

43 

36 

83 7 

57.4 

1-99 

121-125 

178 

21.3 

52.1 

1-99 

74 

57 

77.0 

53.3 

5-100 

116-120 

179 

2L+ 

45 5 

1-98 

71 

52 

73 2 

42.4 

2-91 

111-115 

119 

142 

41 3 

1-97 

48 

32 

66 7 

38.5 

2-95 

106-110 

75 

9.0 

33 3 

1-94 

39 

26 

66.7 

31.1 

5-65 

101-105 

29 

3.5 

36.7 

2-98 

18 

8 

44.4 

37.3 

4-69 

96-100 

8 

0.9 

25 1 

3-42 

2 

2 

100 0 

27 0 

26-28 

91-95 

5 

0.6 

25,8 

7-69 

2 

2 

100.0 

24 5 

14-35 


835 3+7 253 72 9 



35 45 55 65 75 85 95 105 115 125 135 145 155 165 

44 54 64 74 84 94 10+ 114 124 134 144 15+ 164 17+ 

Figure 1 

Distributions of the IQ*s in the Termati-Menill Standardization Group 
and the Oberlin Gtoup 
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cent exceeding the Terman-Merrill mean. The Oberlin sample 
is rather sharply peaked, showing little kurtosis, but it displays 
a slight positive skewness. Variability is much less than that 
of the Terman-Mcrrill sample, sigma being 10 4 as compared 
with 16.4 for the larger group. 

8. Mean Scholarship of Students of Dif event I.Q. Levels. 
Table 5 reports the numbers and proportions of students of 
the different levels of I.Q. with their (1) mean freshman 
scholarship lank, (2) mean senior scholarship rank, (3) the 
range of scholarship achievement for those at each level, and 
(4) the proportion of those in college long enough to have at- 
tained senior status who did so, for each I.Q. level. Exami- 
nation of the table reveals the following salient facts: 

(a) As indicated by the correlation coefficients previously 
noted, the general tendency is for those of higher I.Q. to 
make the better scholastic records. 

However, (b) the range of scholastic performance is, 
with few exceptions, remarkably similar at each test level. 
Freshman achievement in the highest and the lowest deciles 
is recorded for students with I.Q.’s ranging all the way from 
105 to 140, although no student with an I.Q. below 111 
achieved a top tenth ranking for the entire college course. 
There was one student with an I.Q. of 105 who achieved a 
proportional rank of 98 in freshman scholarship and has been 
in the upper tenth of her class in each of the two subsequent 
years Her centile score, according to state norms, on the 
OSU Test is, however, 71, so the later test is evidently a more 
accurate index of her intellectual ability. 

(c) The four students with I.Q.’s above 150 all made 
exceptionally good records. 

(d) Sufficient time has elapsed to permit but four stu- 
dents whose I.Q.’s are below 101 to become seniors. They 
have all obtained the A.B. degree, but in only one instance was 
this achieved in the normal four-year period. By persistent 
effort, however, they did finish the course, and all of them 
ranked above the lowest decile of their class. This is com- 
parable with Adams’ finding at the University of Texas. 
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(e) The retentive power of the college was not mate- 
rially greater for those at the higher than for those at the 
lower extremes of the distribution 

(f) Some degree of selectivity is indicated, however, by 
a comparison of those with I.Q.’s above 126 with those whose 

I.Q.’s are below 116. Of the 93 in the group with the higher 
I.Q.’s, 74, or 79.57 per cent, persisted, whereas but 70, or 
64.22 per cent, of the 109 with the lower I.Q.’s persisted to 
the senior year. As the ratio of the difference in these pro- 
portions to the standard error of the difference is 2.47, it is 
fairly significant. 

Summary 

1. l.Q.'s were available for 835 cntciing freshmen and 
for 253 of these who had reached the senior year, the scores 
having been derived from the following group tests : Otis, 
Tcrman, Henmon-Nelson, National and Kuhlmann-Ander- 
son. Scores on the OSU Psychological Examination were also 
obtained. 

2. The difference m the power of the diffeient I.Q tests 
to predict college scholarship was not statistically significant. 

3. The I Q ’s constitute a better basis for predicting col- 
lege grades than they do for prognosing total high school 
scholarship. This is also true of the OSU Test scores, 

4. The OSU Test is more successful than any one of the 
other tests in predicting scholarship in high school as well as 
in college. 

5. The OSU Test taken during Freshman Week corre- 
lates more closely with both secondary and college scholar- 
ship than does the same test taken during the senior year 
in high school. 

6. The OSU Test predicts freshman scholarship better 
than it docs total college scholarship. 

7. The average I.Q. of the freshmen is 121. The aver- 
age obtained by the Otis, Terman, and Henmon-Nelson tests 
is virtually the same. The averages for the small number 
tested with the National and Kuhlmann-Anderson tests are 
127 and 125, respectively. 


396 



VALUE OF I.Q.’S FOR PREDICTING COLLEGE SCHOLARSHIP 


8. The average I.Q. for the seniors is also 121. As the 
mean OSU Test score for the seniors is but one percentile 
point highci than that for the freshmen, it is evident that 
virtually no selection occurs during the college course, so far 
as test intelligence is concerned. To be sure, there is some 
selection in terms of scholastic lecord, there being a supe- 
riority of 5.46 points in the freshman scholastic rating of 
those who persisted over those who did not become seniors. 
The critical ratio of this difference is 2.73, 

9. The I.Q.’s of the Oberlin freshmen range from 92 to 

169. 99 per cent of the I.Q.’s arc over 100. Variability 

is represented by a sigma of 10 4, as compaied with 16.4 
for the Ternian-Mernll standardization group. 

10. Although the correlation between I.Q. and college 
scholarship Is .40, the range of scholastic performance is re- 
maikably similar at the different test levels between 101 
and 140. 

11. Four students with I.Q.’s between 91 and 100 became 
seniors, but their records were not brilliant. 

12. Although the retentivity of the college was not mate- 
rially greater for those at the extremely high end of the 
distribution than for those at the lower end, 80 per cent of 
those with I.Q.’s above 126, as compared with 64 per cent of 
those with I.Q.’s below 116, who had been in college long 
enough, became seniors. 

Conclusions 

Two facts of general significance emerge from the compu- 
tations : First, the figures indicate that, although it is to be 
expected that students with higher intelligence test scores will 
make the better college records, it is nevertheless possible for 
the average of the group with I.Q.’s as low as 101-105 to do 
acceptable work at Oberlin. There are indeed exceptional 
students who, in spite of the handicap of an Intelligence quo- 
tient as low as 92, obtain the A.B. degree. Second, test scores 
show a consistently closer correlation with college scholarship 
than with high school records. Interpretation of these facts 
would seem to point to the significance of adequate motiva- 
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tion. Possessed of determirLatioii, drive, and directionality, 
the student whose intellectual abdity barely equals the average 
of the general population can “make the grade”, if equipped 
with a good secondary school preparation. Selection of the 
student body at Oberlin is made primardy on the basis of the 
high school record. Students with low I.Q.’s, who rank in 
the lower half of their high school class, are not admitted. 
The higher validity figures obtained when college scholarship 
is used as the criterion also emphasize the factor of motiva- 
tion. Oberlin students, at any rate, apparently work more 
nearly up to their potential capacity, so far as this is measured 
by the intelligence tests, while in college than in secondary 
school. 
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THE THURSTONE PRIMARY MENTAL ABILITIES 
TESTS AND COLLEGE MARKS 

MARY LOU ELLISON 
and 

HAROLD A. EDGERTON 
Ohio Stafe University 

T he present study of Thurstone’s Primary Men- 
tal Abilities Tests has been made in order to implement 
the assumption that the scores of the several factors might 
be useful in academic counseling. Four questions form the 
basis for the investigation. 

1. What relationships are there between the factor scores 
and academic grades? 

2. What relationships are there between the Ohio State 
University Psychological Test score and the factor scores? 

3. How well can academic grades be predicted on the 
basis of the primary factor scores? 

4. Are the factor scores related to grades in specific col- 
lege subjects? 

Thurstone’s development of his Primary Mental Abilities 
Tests was for the purpose of appraising seven primary fac- 
tors of mind ^ His isolation of these factors and the devel- 
opment of the final test battery is described in the monograph 
“Primary Mental Abilities.’’^ Thurstone briefly describes 
the factors on his individual record sheet for the tests as 
follows ; 

"Factor P, The tests that call for this ability require the quick 
pciception of detail in either visual oi verbal material. This seems 

IL. L Thurstone, Manual of hisinuiions fot Administering Tests for ?ti- 
mary Mental Ahiliites, 2. 

2L. L. Thurstone, “Pnraaiy Mental Abilities,” Psychometric Monogtaphs 
Chicago: The Univer.sity of Chicago Press, 1 (1938), 
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to be a perceptual ability which enables some people to excel in 
finding detail which is significant to them or detail which they arc 
seeking. It Is probably one of the factors that is involved in what 
has been called ‘quick intelligence.’ Scanning a page to find quickly 
some small but significant detail and classifying familiar objects 
(|iiickly are examples of this factor. 

'"Facto? N This is one of the clearest factois that has been 
isolated. It consists of facility with simple numerical Avoik and is 
best leprescnted in the tests of lapid calculation. It is of secondary 
impoitance in arithmetical leasoning and in decipheiing numeiical 
code, tasks which call foi factors in addition to facility with num- 
bers as such. It is not yet known whether this factor can be exem- 
plified in non-numerical tasks. 

'^Factor' F. This is a verbal factor which is manifested in tests 
that involve the interpretation of language. It rs not restricted to 
mere fluency witli words. It reflects an ability to deal readily and 
quickly Mith verbal material. Those who excel in this factor arc 
probably verbally-minded in their thinking and problem-solving. 

"Factor S. This is an ability that is present in those tests which 
lequirc the subject to think visually of geometrical forms and of 
objects in space. While none of these factors can be described in 
detail yet, it seems reasonable to expect that those who have a high 
rating on ability S should be able to do well in those studies and in 
those occupations that require visualixdng or thinking about things 
in visual form. Many people think about a problem visually even 
when the nature of the problem docs not immediately suggest any 
necessary visual character. 

"Factor M. The nature of this factor was identified by the 
fact that all of the tests which require It are tests of memorizing. 
The appearance of sucli a factor seems to give justification for the 
belief that a good memory is an ability independent of other mental 
powers. It is not yet known, however, whether the ability to 
memorize is the same as the ability to recall experiences which we 
do not intend to retain for future recall. The present factor M can 
be tentatively named the ability to memorize. 

"Factor L The tests whicli require this factor demand that 
the subject discover some rule or principle in the mateiial of the 
test. Tile factor does not seem to be restricted to mateiial which is 
primarily numerical, primarily visual, or primarily verbal, types 
which were all represented in the tests for this factor. The ability 
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to discovei a iiilc oi principle in the solution of a pioblem is usuall)^ 
called Induction, People differ markedly in the kind of lesource- 
fulness that is involved in inductive thinking, and the hypothesis that 
the factor I is associated with this kind of ability seems plausible. 
It Is not known whether this factoi is associated with inventiveness 


and initiative. 

'"Factor D. The deductive factor is still only tentatively iden- 
tified. It is a factoi which is present in syllogistic reasoning and 
also in some other tests. It is one of several factors that may be 
involved in restiictive thinking. In a general description, the factoi 
seems to represent facility in foimal reasoning^ 

In the present study, Thurstone's Primary Mental Abili- 
ties Tests, Experimental Edition were used 

The subjects consisted of a group of 49 students in the 
College of Arts and Sciences, Ohio State University. Most 
of those who took the test were students in the Exploratory 
Program of the College of Arts and Sciences. 

The students tested do not constitute a random sample 
of students of the Exploratory Program, nor of the College 
of Arts and Sciences, nor of freshmen generally. This fact 
must be taken into consideration in the interpretation of the 
results of the study. No one was required to take the test. 
Of the forty-nine subjects, forty-one were freshmen, six were 
sophomores, and two were juniors In the group, 39 per cent 
ranked in the 90th percentile or above in intelligence (Ohio 
State University Psychological Test), and 54 per cent were 
included in the 80th percentile or above. The mean Point 
Hour Ratlo^ was 2.40. 

In addition to the scores for the seven factors, and the 
separate scores on the sixteen individual tests from which 
the factor scores are derived, other data from the college rec- 
ords were used. Intelligence test percentiles were based on 
scores received in the Ohio State University Psychological 
Examination, given to all students at the time of entrance 


8The Point Hour Ratio is the total points divided by the hours 

For each hour of grade A, four 

points; C, two points; D, one point, and E (fai )i 


attempted 
of B, three 
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to the University. The Point Hour Ratio for each student 
was obtained Grades received in English, sciences, foreign 
languages, and psychology were recorded, since it was thought 
that each of these groups might be related differentially to 
the factor scores. In the group of forty-nine students, English 
grades were available for twenty-seven, science grades for 
thirty, foreign language grades for twenty-seven, and psy- 
chology grades for twenty-five. 

1. What relationships arc there between the factor scores 
and Point Plour Ratiof 

In Table 1, the correlations between Point Hour Ratio 
and the various factors are shown. The correlation between 
Factor V and Point Hour Ratio is the highest (0.44). Fac- 
tor M ranks second in its correlation with P. H. R., the corre- 
lation being 0.31. The other five factors have correlations 
with P. FI. R. ranging from — 0.24 to 0.19. One might specu- 
late on the meaning of the negative correlations, but on the 
basis of such a sample it might be unfortunate. 

It is likely that in a really random sample of University 
students or of University freshmen such correlations would 
be zero Or positive. 

The multiple correlation between P. FI. R. and the 
weighted scores of the seven factors is 0.640. When the 
Ohio State University Intelligence Test score is included as a 
variable with the seven factors, the multiple correlation is 
0.648. Such a correlation suggests that there may be some 
justification for the use of the Primary Mental Abilities Tests 
for the prediction of academic success in college. 

2. JVhat relationships are there between the Ohio State 
University Psychological Examination scores and the factor 
scores? 

As in the case with Point Hour Ratio, Factor V shows 
the highest correlation with intelligence (0.S2). This is 
perhaps due to the fact that the expression of intelligence is 
largely verbal in character in present tests. The Same- 
Opposite Test, a component of Factor V, shows the highest 
correlation of the several sub-tests with intelligence test 
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'PABLE 1 

COMPARISON OF CORRELATIONS OF FACTORS AND INDIVIDUAL TESTS 
•WITH INTELLIGENCE TEST SCORES AND POINT HOUR RATIO 

(N^49) 

Intelligence Test Point Hour Ratio 


Composite Individual Composite Individual 
Facto i Test Factor Test 


Factor P 

0.06 


-0.24 


Identical Forms 


-0 05 


-0.21 

Verbal Enumeration .... 


0.16 


-0.27 

Factor N 

-0.02 


0.17 


Addition 


-0.07 


-0.07 

Multiplication 


0.01 


0,29 

Factor V 

0.52 


0.44 


Completion 


0.41 


0.33 

Same-Opposite 


0.55 


0.37 

Factor S 

-0 11 


-0.21 


Figures 


-0.12 


-0.40 

Cards 


-0 07 


-0 01 

Factor M 

0.28 


0.31 


Initials 


0.34 


0 32 

Word" Number 


0.09 


0.17 

Factor I 

0.11 


-0.13 


Letter Grouping 


0 24 


-0.18 

Maiks 


0.04 


-0.32 

Number Patterns 


0.07 


-0.23 

Factoi D 

0.10 


0.19 


Arithmetic . . 


0.09 


0.10 

Number Scries 


0.36 


0.35 

Mechanical Movement . . 


-0.13 


0 04 


scores. A somewhat similar test is found in the Ohio State 
University Psychological Test. 

The correlation of Factor M with intelligence is 0.28. 
The Initials Test correlated 0.32 with intelligence, while the 
other component of Factor M, the Word-Number Test, cor- 
related very low (0.09). 

The correlation of factors P) !> ^ intelligence 

are positive, but are very low. Among the components of 
Factor D, the Arithmetic Test has a low correlation with 
intelligence (0.09), the Number Series Test has one of the 
highest correlations in the battery with intelligence, an t e 
Mechanical Movements Test shows a negative corre ation 
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TABLE 2 


INTKRCORRI'I.ATIONS BETWEEN POINT HOUR RATIO, PSYCHOLOGICAL 
I'KST, AND THE SEVEN FACTORS 

(N =49) 




(IntelliRcnce) 

u.s.u. 

Psych, 

P.II.R. Test P 

N 

V 

« 

M 

I 

D 

p. n. 

K.. 


0.20 

"0,2+ 

0.17 

0.44 

-0.21 

0.31 

-0.13 

0.19 

0 . s. u. 










Psych. 

Test 0.20 


0 06 

-0.02 

0.52 

-0.11 

0,28 

0.11 

0.10 

P 


-0,2+ 

0,06 


0.3+ 

0 27 

0 54 

0.14 

0 47 

0.10 

N . 


0 17 

^0,02 

0.3+ 


0.43 

0 18 

0.28 

0,38 

0,42 

V . . 

, 

0.44 

0,52 

0 27 

0 +3 


0 12 

0.33 

0 36 

0.34 

S 


-0,21 

-0.11 

0.5+ 

0 18 

012 


0,08 

0,34 

0.19 

M . 


0 31 

0.28 

0.14 

0.28 

0 33 

0 08 


0.13 

014 

I ... 


-0.13 

0.11 

0,47 

0.38 

0.36 

0.34 

0.13 


017 

D . 


0.19 

0 10 

0.10 

0.42 

0.34 

0 19 

0.14 

0.17 


Mean 


2 40 

71.6 

139.9 

107.1 

76.9 

108.7 

15 1 

30.7 

72 

Standard 










Devintinn 

0.59 

20.3 

25.1 

36.4 

23.8 

3 6.4 

7,6 

7.8 

2.4 

with 

intelligence. 

Such 

correlations 

might 

raise 

a question 


regiirding the functional unity of the factors 


3. How well ran Point Hour Ratio be -predicted on the 
basis of the primary facial scores? 

It would be desirable to be able to predict the probable 
P. tl. R. of a student from the scores made on the seven 
factors. The chart below shows that in both situations, the 
highest beta weight Is that for Factor V. 

I'ABLE 3 

BETA AND b REGRESSION COEFFICIENTS 

For the Scoies When the OSU Psychological Test is Included 
and When It Is Omitted From the Test Battery 

OSn Intelligence Included OSU Intelligence Omitted 


Beta b Beta b 

Coefiicient CoefficienL' Coefficient Coefficient 

Factor P -.279 -.007 -.291 -.007 

Factor N 034 .001 .090 .001 

Factor V 568 .014 .487 .012 

Factor S -.113 -.002 -.089 -.001 

Factor M 216 ,017 .191 .015 

Factor I -.196 -.015 -.201 -.015 

Factor D 046 .045 .040 .039 

OSU Psychological 

Examination -.136 —.004 
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The correlation with P. H. R. is increased slightly when 
the intelligence test rating is used, the correlation between 
P. H. R. and the variables being raised from 0.640 to 0.648. 
In a random sample of freshmen this difference would prob- 
ably be greater. 

4. Are the factor scores related to grades tn specific 
college subjects? 

The correlations of course grades with Point Hour Ratio, 
intelligence test scores, and the seven factors are found in 
Table 4. The grades taken into consideration in this study 
are those in English, science, foreign languages, and psychol- 

TABLE 4 

THE CORRELATION OF SUBJECT MATTER GRADES WITH POINT HOUR 
RATIO, INTELLIGENCE, AND THE SEVEN FACTORS 


Foreign 

English Science Language Psychology 

Grade Grade Gi ade Grade 


P. H, R 

IHE^ 


0.77 

0.58 

Intelligence 

HQy 


0.54 


Factor P 

HIKIISI 

-0.12 

0 27 

■■EIQH 

Factor N 

0.34 

0.03 


0.37 

Factor V 

. 0.75 

0.68 


0.59 

Factor S 

. . 0 44 

0.23 

0.56 


F actor M 

, 0.42 

0.18 

0.45 

0.23 

Factor I 

. , 0.24 

0.05 

0.78 


Factoi D 

0.44 

0.23 

0.43 

0.63 

Number of Cases. . . 

. 27 

30 

27 

25 

ogy. The results 

must be 

taken as 

suggestive 

and not as 


facts from which broad generalizations may be drawn. 

In all four cases, there are high correlations between P. 
H. R. and grades. This is to be expected, since these grades 
are components of the Point Hour Ratio. 

English grades correlate highest with Factor V (0.75). 
Factors S, M, and D also show correlations above 0.40 with 
English grades. 
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The only factor showing a correlation above 0.40 with 
science grades is factor V. 

All the factors are apparently important in determining 
foreign language grades, since all factoi-s except Factor P 
correlate above 0.40 with foreign language. The most 
significant correlation with foreign language is Factor I 
(0.78). This correlation is higher than one would expect, 
but It may be due to the fact that an inductive method is used 
at Ohio State University in teaching the beginning language 
courses. 

The highest correlation among the factors with psychol- 
ogy grades is with Factor D (0.63). Factor V is also high 
(0.59). These were the only two factors correlating above 
0.40 with psychology. 

Factor V correlates above 0.40 with grades in each of 
the' four subject fields considered, the highest being with 
English grades. The correlations between Factor I and the 
school subjects are low with the exception of foreign lan- 
guage grade (0.78). Factor P shows very low correlations 
with all four school grades. There is little differentiation 
between the correlations of the school grades and Factor N, 
the only correlation higher than 0.40 being with foreign 
language grade. Factors S and N both have correlations over 
0.40 with English and foreign language grades, and Factor D 
has a significant correlation with English, foreign language, 
and psychology grades. 

Such observations as reported here suggest that, with 
more experience, the Thurstone Primary Abilities Test will 
become a useful instrument in the academic counseling program 
of colleges. It will be necessary to secure more data in regard 
to the relationships of test scores and course grades from a 
random sample of freshmen. Also, it will be important to 
have some knowledge of methods of instruction in the several 
courses so as to judge whether the relationship observed is a 
function of the abilities of the student and the subject matter 
being studied, or of the methods of instruction. 
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A SHORT CUT IN THE ESTIMATION OF 
SPLIT-HALVES COEFFICIENTS 


CHARLES I. HOSIER 
Social Security Board 

F or SEVERAL YEARS the writer has availed himself 
of a short cut in the computation of reliability co- 
efficients by the split-halves technique. The method has prob- 
ably been developed independently by a number of other 
investigators, but it has not, to the writer's knowledge, 
appeared in print in connection with this specific problem, 
and there may be some workers to whom it may prove useful. 

With the development of the Kuder-Richardson method 
for the determination of reliability, the split-halves technique 
should probably disappear from the scene. However, as a 
number of investigators have found, it provides a fairly close 
approximation to the Kuder-Richardson value, and since it 
does not require an item-analysis it will probably continue in 
use. In any event, the purpose of this note is not the justifica- 
tion of the technique, but the presentation of a short cut. If 
split-halves coefficients are to be computed, they may as well 
be computed efficiently. 

In brief, the short cut involves the use of the complete 
dependence of the “even” scores on the “total” and the “odd” 
scores. We may suppose that “total” scores have already 
been obtained in connection with the original purpose of the 
test. Because of this algebraic dependency, then,^ it remains 
only to rescore the papers for the “odd” scores in order to 
know the even scores, since, 

A, - Y. = 0, (1) 

Furthermore, equation (1) need not be applied to each case 
separately. Not only may we dispense with the necessity of 
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rescoring the papers to get the “even” score for each indU 
vidual; we may go farther, and dispense with the necessity 
of obtaining the individual “even” scores at all. 

By defining the value desiied, namely, the correlation be- 
tween O and E, and substituting the value of E expressed as 
a function of O and T from equation 1, we obtain the result 
that 


^'nr tTj Oq 

(Tji "h Cq ~~ 2? Qji Oq Oji 

This expression calls for only the “odd and “total” scores, 
from which are obtained their respective standard deviations 
and the correlation between them. The value obtained from 
the odd-even correlation can then be used in the Spearman- 
Brown formula to give the estimated reliability. This value 
Is identical with that which would be obtained if each test 
paper were independently scored for the “even” score, and 
the odd-even correlation coefficient computed from the re- 
sulting data. (This can be seen from the derivation.) 

The expression in equation (2) is readily recognized as 
a special case of the mote general formula for a correlation 
between a part and the whole exclusive of the part. 0, E, 
and T in equation (2) are by no means limited to “odd,” 
“even” and “total” scores, but apply to any set of variables 
for which equation (1) is true. 
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Adams, C, R. “A New Measure of Personality.” Journal of 
JppUed Psychology, XXV (1941), 141-151, 

A new instrument for measuring personality traits, The 
Personal Audit, is described. It was intended to be relatively 
free from highly personal items since it was felt that such a 
test would be more useful in non-clinical situations. The Per- 
sonal Audit is believed, on the ba^is of low intercorrelations 
between sub-tests, to measure 9 relatively independent person- 
ality traits. Coefficients of reliability (corrected split-half) 
range from +.90 to +.96. Items have been validated by the 
criterion of internal consistency using a modified version of 
the Sletto technique JP. A. Varvel. 

Baxter, B. and Paterson, D. G. “A New Ratio for Clinical 
Counselors.” Journal of Consulting Psychology, V 
(1941), 123-126. 

The magnitude of the S.E.j^, which clinical counselors 
employ in interpreting test scores, varies in significance with 
the variability (S D.) of the norm group. It is useful, there- 
fore, to relate them in a ratio as an aid in interpreting 
scores. The following formula, in which r is the reliability 
coefficient, provides a simple way of expressing the magni- 
tude of as a percentage of S.D . . 

S.E.^ S.D, 

TDr= — m — = 

Application of this ratio to a list of 49 tests shows that 

ranges from as low as .10 to as high as .55. In gen- 

S.D. . . ^ 

eral, achievement tests show the lowest ratio (highest accur- 
acy) with an average of .20, followed in order by scholastic 
aptitude tests (averaging .30), reading tests (.32), special 
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aptitude tests (.33), and personality tests (.27 to .55, aver> 
age .40). F. A. Kingsbury, 


Bennett, G. K. and Raskow, S. “Extension of the Norms of 
the Columbia Vocabulary Test ’’ Journal of Applied 
Psychology, XXV (1941), 48-51. 

Constructed and standardized for grades 3-8, this test 
showed a mean score of 54 and standard deviation of IS for 
the latter half of the eighth grade. Extension of norms seems 
justified when 1212 superior recent high school graduates 
obtained a mean score of 74 with standard deviation of 12. 
When the test was administered to 5101 high school students, 
mean scores increased and the standard deviation decreased 
from grade 9A through grade 12B both among commercial 
and general-course students. Decreasing standard deviation 
probably indicates increasing homogeneity of vocabulary in 
later school years. With grade constant, mean scores decrease 
with age. J. E. P. Libby. 


Blum, Milton L, and Candee, Beatrice. “The Selection of 
Department Store Packers and Wrappers with the Aid of 
Certain Psychological Tests; Study II.’’ Journal of Ap- 
plied Psychology, XXV (1941), 291-299. 

This study is a check on conflicting results in previous 
attempts to determine the value of finger dexterity tests in 
predicting successful wrappers or packers. Tests used were 
the O’Connor P'Inger Dexterity, Zeigler Placing, Otis Self- 
Administering, and Minnesota Clerical. Test performance was 
checked against production records and foreman’s ratings. 
Results indicate no relation between finger dexterity and pro- 
duction for cither packers or wrappers. In the experienced 
group the Minnesota Clerical shows positive correlation for 
both groups. It is concluded that clerical speed and accuracy 
have a much higher relation to production than has finger 
dexterity. D. A. Peterson. 
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Brown, A. W. and Blakey, R. “A Preliminary Report on the 
Development and Standardization of a Non-Verbal Test 
at the High-School Level.” Journal of Educational Psy- 
chology, XXXII (1941), 113-123. 

A series of 11 non-verbal subtests constructed on the con- 
cepts of primary mental abilities has been standardized on a 
group of 286 suburban high school students, Eight of these 
subtests, two of perceptual speed, two of spatial relations, and 
four of abstract reasoning, constitute the final test, which may 
be given in forty minutes. The “Non-Verbal Reasoning Test” 
correlates with school grades .47 and with Otis I.Q. .59; Otis 
I.Q. correlates with school grades .60. Higher correlation 
with grades was not expected since the latter involve other 
abilities in addition to those in the non-verbal test. Reliability 
of the test is .97. Tentative norms are given, including derived 
scores intended to take the place of I.Q.’s at this level; stand- 
ardization on a much larger sample is being undertaken. /. E. 
P. Libby. 


Brown, A. W. and Cotton, C. B. “A Study of the Intelligence 
of Italian and Polish School Children from Deteriorated 
and Non-Deteriorated Areas of Chicago as Measured by 
the Chicago Non-Verbal Examination.” Child Develop- 
ment, XU {19^1) , 21-30. 

1262 Italian and Polish school children in a deteriorated 
and a non-deteriorated area were tested with a non-language 
group test battery. The children were in the fourth grade or 
above and from 10 to 14 years of age. The authors stress the 
influence of socio-economic, cultural, and educational factors 
upon test scores. They found (1) a regular decrease in mean 
test performance from age 10 to age 14 for both sexes and 
both nationality groups but not so great as that previously re- 
ported for verbal tests; (2) sexual differences favoring the 
boys, particularly in the case of Italian children; and (3) 
contradictory indications relating to socio-economic community 
level (no significant differences between areas for Italian 
boys; significant differences favoring the deteriorated area for 
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Italian girls; a tendency for Polish children in the non- 
delcrioratcd area to make better scores). W. A. Faivel. 


Brush, Edward N. “Mechanical Ability as a Factor in Engi- 
neering Aptitude.” Joiimal of Applied Psychology, XXV 

(1941), 300-312. 

This study was Intended to explore the possibilities of 
available tests of mechanical ability and aptitude as Indicators 
of aptitude for engineering. The report is prefaced with a 
survey of the relevant literature. The subjects were two groups 
of students in the College of Technology at the University of 
Maine, one group of 104 members, the other group of about 
130 members. The criterion was scholastic rank in courses of 
an engineering nature. The tests used were : Minnesota Paper 
Foim Board, Minnesota Assembly Test, Minnesota Spatial 
Relations 'Pest, O’Connor Worksample No. 1, O’Connor 
Worksample No. 5, O'Connor Worksample No. 72, Cox 
Mechanical Explanation and Completion Test, Cox Mechani- 
cal Models d’est, and MacQuarrle Test for Mechanical 
yVbility. In addition data on intelligence tests, algebra, chemis- 
try, plane geometry, and physics tests were also available. 

The conclusions reached are summarized as follows : “The 
tests of useful predictive power were the Cox Tests of Mechan- 
ical Aptitude and Minnesota Paper Form Board .... Bat- 
teries of mechanical ability tests yield correlations with the 
criterion of about .40; batteries in which an intelligence test is 
combined with one or two tests of mechanical ability yield 
correlations of about .50 ... . seveial batteries of mechanical 
ability tests predict engineering scholarship at least as well as 
the intelligence tests, while the achievement tests, singly and in 
combination, predict success in engineering studies somewhat 
jbetter than do the tests of mechanical ability .... total 
engineering record is more highly correlated with first 
semester and first year grades than with any test or combina- 
tion of tests.” /. E. Karlin. 
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Burtt, H. E. and others. “Market Problems and Market Re- 
search.” Journal of Consulting Psychology, V (1941), 
No. 4, 145-193. 

This entire number Is devoted to eight papers on market 
research, not separately abstracted here because of space 
limitations. The authors and titles are as follows: “Current 
Trends in Marketing Research” (H. E. Burtt) ; “Proving 
Ground on Public Opinion” (H. G. Weaver) ; “Problems of 
Sampling in Market Research” (Frank Stanton) ; “Charac- 
teristics of the Question as Determinants of Dependability” 
(J. G. Jenkins) ; "Evaluating the Effectiveness of Advertising 
by Direct Interviews" (P. F. Lazarsfeld) ; “Effects of Re- 
peated Interviewing on the Respondent’s Answers” (F. D. 
Ruch) ; “The Museum Technique Applied to Market Re- 
search” (G. K. Bennett) ; and “The Role of Psychological 
Interpretation in Market Research” (A. W. Kornhauser). 
F. A. Kingsbury. 


Casanova, T. “Analysis of the Effect upon the Reliability 
Coefficient of Changes in Variables Involved In the Estima- 
tion of Test Reliability.” Journal of Experimental Educa- 
tion, IX {mi) , 219-22S. 

The following topics are discussed and various formulae 
developed in detail; (1) the variance of the halves in the 
split-half method of estimating reliability; (2) the correction 
for guessing with specific reference to the reliability of rights 
and wrongs, the variance of rights and wrongs, the correlation 
of rights with wrongs, the variance of the number of items 
attempted, and the number of possible choices; (3) the effect 
of calling all negative scores zero; (4) the variance of the 
items. In the latter case, a formula for estimating the re- 
liability of a test in terms of the item variances is presented 
which is felt to be more convenient than the Kuder-Richardson 
formulae. hF. R. V arvel. 


Cattell, Raymond B., Feingold, S. Norman, and Sarason, Sey- 
mour B. “A Culture-Free Intelligence Test : 11. Evaluation 
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of Cultural Influence on Test Performance.” Journal of 
Educational Psychology, XXXII (1941), 81-100. 

A culture-free Intelligence test described In an earlier paper 
was administered together with the Blnet (Terman-Merrlll), 
A.C.E. (arithmetical sections), and the Arthur Performance 
Tests, to four comparable groups; these were given special 
training, one group in each class of information or skill de- 
manded by the tests. Retest analyses showed the Arthur least 
influenced by training in its own culture medium, the Culture- 
Free Test next, Binet next, and A.C.E. most influenced. The 
above tests and the Ferguson formboards were administered 
to a group of adult immigrants, resident in this country about 
one year, and a control native group. The Ferguson was very 
close to the Arthur, others followed in the order noted earlier, 
when the groups were retested after 77 days during which the 
immigrants gained noticeably in Americanization. Reliability 
of the Culture-Free Test compares favorably with those of the 
others. Adequate validity is indicated by the Culture-Free 
Test’s high loading in the general factor brought out by 
tetrads, and by its high mean correlation with the pool of 
tests. Since life experience probably brings factors in the Cul- 
ture-Free Test to saturation in widely different cultures, its 
proper application appears broader than that of preceding 
tests. J. E, P, Libby, 


Driver, Randolph S. ‘‘The Validity and Reliability of Rat- 
ings.” XVII (1941), 185-191. 

Rating is of value in industry only when its limitations as 
a scientific instrument are fully appreciated. The various cur- 
rent methods of obtaining measures of validity and reliability 
are discussed and their values and limitations considered. In 
order for a rating to be acceptable, it must be proven valid 
and reliable. Although difficult to accomplish, ratings are not 
useless, but great caution must be observed in their interpreta- 
tion. Virginia Brown. 
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Dudycha, George J. “A Suggestion for Interviewing for De- 
pendability Based on Student Behavior.” The Journal of 
Applied Psychology, XXV (1941), 227-231. 

College students were divided into groups of extreme 
earliness and lateness, of dependability and undependability, 
on the basis of observation of their behavior in life situations. 
Ten questions on punctuality and persistence and six on de- 
pendability, when presented to these contrasting groups, 
elicited responses indicating significant group differences. Since 
these questions appear to be diagnostic in student behavior, it 
is suggested that they be tested for usefulness in employment 
situations for discovering those applicants likely to prove 
dependable. Virginia Broivfi. 

Dulsky, S. G. “Vocational Counseling. I. By Use of Tests; 
II. By Interview.” Personnel Journal, XX (1941), 16-28. 

The author briefly and critically examines various types of 
standardized tests available to the vocational counselor. He 
concludes that aptitude tests are of no value and personality 
tests of very limited value. Interest inventories, if used prop- 
erly, may be helpful. Tests of intelligence and educational 
achievement are approved as being of the most value. He 
advocates greater emphasis on the vocational interview as a 
means of diagnosing personality and motivation and of identi- 
fying and evaluating interests. Self-guidance from the study 
of test scores and profiles is impossible. Vocational counseling 
is an individual process, requiring “skilled psychologists” 
rather than “mental testers.” The vocational counselor should 
confine himself to descriptive rather than quantitative reports 
of test and interview results and should only rarely go beyond 
general recommendations to his clients. W. A. VarveL 

Ebert, Elizabeth H. “A Comparison of the Original and 
Revised Stanford-Binet Scales.” The Journal of Psychol- 
ogy, XI (mi) , ^T6l. 

1434 records of 315 children five to ten years of age were 
studied for information as to the comparability of I.Q.’s from 
the original and revised Stanford-Binet Scales. An increasing 
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discrepancy between I.Q. values on the two scales was found 
at ages 7, 8, and 9. The new revision tends to give lower 
LQ.’s for levels below 100 and higher LQ.'s for levels above 
100. The duller children gain slightly more in I.Q. than the 
brighter ones although both groups show increases. With the 
old revision, the duller individuals gain but the brighter ones 
lose. Although the average I.Q. of the 1916 revision was more 
constant, individuals maintained their relative positions better 
in the new revision. J’irginia Brown. 


Eysenck, 11. J, “Type-Factors in Aesthetic Judgments.” 
British Journal of Psychology, XXXI (1941), 262-270, 

It has been found previously that the analysis of the inter- 
correlations between the rankings of pictures by a number of 
subjects yields mainly one general factor with no other sig- 
nificant factor. On this occasion the attempt is made to bring 
out the influence of any such secondary factor, even, if need 
be, at the expense of the "T” or general factor. Five series 
of pictures, each consisting of thirty to fifty items, were judged 
in order of goodness by fifteen subjects. The subjects were 
artists, university students, bank clerks, typists, and teachers, 
eight women and seven men, with age range from 20 to 70. 
The table of correlations for each of the five series was fac- 
tored and two significant factors extracted in all cases except 
one. One factor was the “T” factor previously identified; the 
other factor, called the “K” factor, seemed to divide the 
population into two different “types,” one preferring the mod- 
ern, and the other the older style of painting. This factor, 
identified provisionally with “brightness,” correlated with 
extroversion, radicalism, youth, and possibly with preference 
for color, The color-form test also appeared to be correlated 
with extroversion. Results arc definite enough to suggest that 
further research into the relation between temperament and 
aesthetic preferences will not only extend knowledge of the 
“type” factors in aesthetic judgments, but also increase under- 
standing of temperamental “types.” J. E. Karlin. 
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Ferguson, L. W. “A Study of the Likert Technique of Atti- 
tude Scale Construction.” Journal of Social Psychology, 
XIII (1941), 51-57. 

The suggestion is here examined that Likert’s method of 
constructing and scoring attitude scales gives results as valid 
as those of the method outlined by Thurstone and Chave with 
much less labor. Items constructed by the former method 
(Minnesota Scale for the Survey of Opinions) were rescaled 
by the latter; standard deviations of the distribution of scale 
values indicate that such items are adequately scaled by this 
method. The scale values obtained indicate that Likert’s 
technique does not obviate the need of a judging group With 
one exception, the scales cannot be scored by the Thurstone 
method. Scores obtained by the two methods for the excep- 
tional scale show a correlation of .70, confirming the conclu- 
sion. J. E. P. Ltbby. 

(Jrcene, E. B. Measurements of Human Behavior. New York; 
The Odyssey Press, pp.777. 1941. 

This volume of 24 chapters is divided into three parts' 
Part I, “Basic Considerations” (discussing introductory con- 
cepts, varieties of appraisals, score-interpretation, measures of 
relationship, types of instruments, item construction and 
evaluation, factor analysis) ; Part II, “Instruments and Re- 
sults” (tests of early childhood, of achievement, Binet-type 
and group intelligence scales, performance, mechanical and 
motor tests, measures of fine arts— design, literature, and 
music — tests of interests, attitudes, adjustment) ; Part III, 
“Persistent Problems” (effects of practice on scores, measures 
of growth and senescence, absolute scaling, evaluation of judg- 
ments, native differences). A 30-page bibliography, a com- 
bined glossary and subject-index, 121 tables, and 108 figures 
are features of the book. F. A. Kingsbury. 


Guilford, J. P. "A Note on Dubois’s Method of Deriving an 
Achievement Ratio for Students.” Journal of Educational 


Psychology, XXXll {m\), 220-222. ^ 

Dubois’s achievement ratio is that of the student s actual 


417 



KDUCATIUNAI. AXD I'SVCIIOLOGICAL MEASUREMENT 


average mark to that mark corresponding to the standard score 
obtained on a psychological test; these ratios in general are 
low for students with high test scores and high for those with 
low scores. 7'his finding follows from the assumption of a 
correlation of 1.00 between test scores and marks, while 
Dubois gives the correlation as .442. A student may be ex- 
pected to deviate from the mean mark by only 442 as much 
as his standard score indicates. It is suggested that Dubois's 
conclusion might be reversed if the regression line r equals 
.442 he taken as base. Computation of a special but meaning- 
ful case confirms this suggestion. J. E. P. Libby. 


Hay, Kdward N. ‘'Tests in Industry.” Personnel Journal, 
XX (1941), 3-15. 

This 1.S a discussion of the opportunities for psychologists 
in industry. The use of intelligence tests is coming more and 
more to act as a check on employer’s judgment, which is cus- 
tomarily biased in favor of the qualities of aggressiveness and 
good personality. Such tests Indicate the level at which the 
employee is able to work most efficiently and his potentialities 
for further promotion. It is particularly important to obtain 
psychological information about an employee at the time of 
his entry into a firm since the work he does then determines 
to a large extent his opportunities for advancement. A be- 
ginning job may require an l.Q. of about 100 but higher 
positions require higher I.Q.’s so that an employee progress- 
ing reasonably well in the initial job may become unfit when 
advanced to the more complex positions. It becomes advisable 
to judge prospective employees not on the basis of the intel- 
ligence required for their first positions but for the positions 
to which they should be able to rise. With the use of objective 
tests, information becomes generally available for an entire 
firm so that transfers and promotions from one department to 
another can be advised with a minimum of further consulta- 
tion, since the qualities required in other work arc known and 
the abilities of the employee are likewise known at the time of 
first testing. Apart from the question of job maladjustment 
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there is a further fruitful field for the industrial psychologist 
in the problem of a better supervisor-employee relationship 
An illustrative study of these methods at work accompanies 
the discussion. /. E. Karlin. 

Johnson, Donald M. and Reynolds, Floyd. “A Factor Analysis 
of Verbal Ability.” Psychological Record, IV (1941), 
183-195. 

The literature on problem solving among animal and 
human subjects suggests that there may be two fundamental 
processes involved: “F,” the flow of various acts or responses; 
and “S,” the selection of these responses according to the 
requirements of the problem. This study tested the hypothesis 
that individual differences in these two processes is a major 
determinant for scores on problem-solving tests. This investi- 
gation was limited to verbal problems. There were ten tests 
involving the supplying of verbal responses ; the tests varied 
in restriction of choice of responses from complete freedom to 
supply any word to restriction to the supplying of only certain 
words according to a rigid criterion. The subjects were 113 
summer-school students at Fort Flays Kansas State College 
A centroid analysis of the table of corrected correlation co- 
efficients yielded two factors. The tests fell within a positive 
manifold, after rotation, indicating two definite factors reason- 
ably identified as the “F” and “S” postulated in the hypothesis. 
It appears that these two factors are probably closely related 
to, if not identical with, Thurstone’s “W” and "V” factors. It 
is concluded that the two processes or functions mentioned 
account to a large extent for the variance in verbal problem- 
solving tests. These findings are further discussed with refer- 
ence to tests of vocabulary, intelligence, and reading. J. 


Karlin. 

Kornhauscr, A. W. and Schultz, R. S. (et al). “Research on 
Selection of Salesman” (and other papers). lonrMdl of 
Applied Psychology, XXV (1941) , No. 1, 1-47. _ 

Five papers read at the Section of Industrial and Busines 
Psychology of the American Association for Applied Psychol- 
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ogy in 1940, toj^cther with an introductory article, are pre- 
sented in this miniher, hut are not separately abstracted 
because of space llniitatioiis. In addition to the introductory 
article (title us above), the authors and titles of the papers 
are as lolbnvs: “Selection of Casualty and Life Insurance 
Atijents’’ (M. A. Bills) ; “Recent Research in the Selection of 
Life Insurance Salesmen” (A. K. Kurtz); '‘A Rcpoit of Rc- 
searcli on the Selection of Salesmen at the Tremco Manufac- 
turinfi; Company” (C) A. Ohniann) ; “Rroccdurcs for the 
Selection of Salesmen fur a Detergent Company” (J. L. Otis) ; 
and “Selection Research in a Sales Organization” (T. M. 
Stokes). F. A. Kinyshitry. 

Lowell, Frances E. “A Study of the Variability of I.Q.’s in 
Retests.” Jotirnal of Applied Psychology, XXV (1941), 
341-356. 

'I’he main purpose of tins study was to seek corroboration 
ol the results obtainetl in Cleveland Public Schools in recent 
years which seemed to show that the I.Q.’s of school children 
tended in certain instances to vary between test and retest 
The data were composed of 1000 cases that had two tests 
only, 1000 cases that had three to.sts, and 1000 cases that had 
four tests, 'riie Terman 1916 revision of the Binet was used 
in all tests. It was found that there arc significant decrements 
in l.Q, both for groups and for chronological age. Further- 
more, the l.Q. range, the chronological age at first test, and 
the interval elapsing between first and last tests may all be 
eliminated as causes for variation in l.Q. on retest. Nor does 
sex influence variations in l.Q. between first and last tests. 
On the average, four times as many cases on retest decrease 
in LQ. as increase. In particular, those cases that increase 7 
or more points on the first retest decrease 5 times as often as 
they increase on the second retest. The data on the first retest 
seem to indicate that the older the child is, the less chance 
there is that his second LQ. will increase. /, E. Karlin. 

McCloy, C. H. “The Factor Analysis as a Research Tech- 
nique.” Research Quarterly, XII (19dl), 22-33. 
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This paper presents an elementary discussion of some 
fundamental concepts and limitations of factor analysis. Par- 
ticular reference is made to its possible uses in the field of 
health and physical education. Specific examples of precautions 
to be taken and the kind of studies to which this type of 
correlational analysis may be applied are given in terms of 
research in physical education. The method has been utilized 
in (1) studies of motor skills, (2) analysis of anthropometric 
data, (3) analysis of cardiovascular variables, and (4) studies 
of character and personality traits. A 17-item bibliography is 
included. fV. A. V arvcl. 


Mosier, C. 1. “A Psychometric Study of Meaning.” Journal 
of Social Psychology, XIII (1941), 123-140 
256 adjectives expressing judgmental relationships which 
could be placed along a favorable-neutral-unfavorable con- 
tinuum were rated on an 11-point scale by college students in 
psychology. Some 140 ratings were obtained for each word 
“Two basic hypotheses . . . are confirmed: first, that the 
meaning of a word may be considered as if it consisted of two 
parts, one constant and representative of the usual meaning of 
the word, and one variable, representative of individual inter- 
pretation in usage and associated context and general usap, 
second, that the frequency with which any particular meaning 
is evoked is describable by the Gaussian Law ” The presence 
of words with two discrete meanings, yielding bimodal fre- 
quency distributions of responses, was noted. The effect of 
adverbial modifiers on the meaning of an adjective was studied 
“A scale with a rational basis has been developed and values 
describing quantitatively the modal meaning and the ambiguity 
of more than 200 adjectives have been obtained.” W . A. 
Farvel. 


Oral Trade Tests— Group Leaders’ Handbook The Per- 
sonnel and Training Section in collaboratimi with the Local 
Office Operations Section and Chicago Occupational Re- 
search Center. Division of Placement and Unemployment 
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Compensation, 205 West Wacker Drive, Chicago, Illinois. 

February, 1941. 18 pp. 

This handbook is designed to instruct the group-leaders 
in interviewing in the construction and use of Oral Trade 
Tests. It is based on Oral Trade Questions, Vol. /, prepared 
by Occupational Analysis Section, United States Employment 
Service Division (not available for general use). There are 
three divisions. History of Oral Trade Questions; Prepara- 
tion of Oral Trade Questions; and Application of Oral Trade 
Questions in Operating Offices. Specific examples of the appli- 
cation of oral trade questions are given. D. A. Peterson. 


Osgood, C. E, and Stagner, Ross. “Analysis of a Prestige 
Frame of Reference by a Gradient Technique.’’ Journal 
of Applied Psychology, XXV (1941), 275-290. 

This study was designed to demonstrate a method for 
analyz,lng a frame of reference, and to investigate the par- 
ticular determinants of the frame of reference known as occu- 
pational prestige. Subjects were required to judge a number 
of occupational stereotypes with respect to a psychological 
“gradient” continuum varying from the description “brains” 
on the one extreme to “brawn” on the other. In a second part 
of the test the judgments were made about persons rather than 
occupations. The subjects were 100 Dartmouth College men, 
students in introductory psychology, 50 of whom filled out the 
“job” form and 50 the "person” form. There were 15 names 
of occupations and each was accompanied by a set of ten 
characteristics. It was found that general rankings for prestige 
correlate on the average highly with median judgments on the 
gradient test, but that the reactions on the job forms were 
significantly different from the person forms. Prestige is 
imputed to occupations per se on the basis of such character- 
istics as hopefulness, being noticed, financial return, brains; 
prestige is imputed to men in specified jobs on the basis of 
brains, leadership, and self-assuredness. Since the conditions 
of the experiment are deemed to exclude the possibility of 
conscious verbalization of a prestige frame of reference, it is 
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concluded that the mere presentation of a set of occupational 
stereotypes for a series of judgments caused the spontaneous 
establishment of a prestige framework which then determined 
in a reliable manner judgments on the specific traits listed. The 
technique is practical and adaptable. J. E. Karlin. 

Powell, N. J. “Check List for Use in Civil Service Objective 
Test Preparation.” Public Personnel Quarterly, II (1941), 
13-16. 

The author has prepared a diagnostic check list designed 
to increase the probability of considering all the major bases 
for appraisal of the test being constructed. Guiding questions 
are listed under each of the construction problerns for examin- 
ing the individual item and the test as a whole. A dual criterion 
IS suggested and iirstructions for use of check list are given. 
It is emphasized that while the degree of correlation between 
test score and job performance is important, it is not the only 
indicator of adequacy of examination. D. A. Peterson. 

Powell, N. J. “Steps in Written Test Construction.” Public 
Personnel Quarterly, II (1941), 73-76. 

The process of constructing a written test is analyzed, 
assuming that examinations are made public (i.e., a test item 
cannot be used more than once). The following general prob- 
lems are treated in outline form: 1. the determination of the 
abilities to be measured; 2, the determination of the test con- 
tent which measures the desired abilities; 3. the allocation of 
emphasis; 4. the preparation of the test items; 5. the arrange- 
ment and editing of the test items; 6. the experimental tryout; 
7. final test copy; and 8. general considerations, with regard to 
test preparation integrity. D. A. Peterson. 

Rcyburn, H. A. uiid Taylor, J. G. “Some F act o r s in Intel- 

ligencc.” British Journal of Psychology, XXaI (1941), 
249-261. 

This study is intended to throw further light on the con- 
troversy regarding the unitary functioning of a general factor, 
g, in tests of intelligence. The material consisted of ten tests 
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purporting to measure some aspect of intelligence, the tests 
being fonnhnards, repetition of digits, repetition of digits 
backwards, matching tests, absurdities, Portcus mazes, arith- 
metical reasoning, leasoning tests, vocuhulary tests, and dis- 
sected sentences. The tests were given tt) 1497 South African 
children with ages ranging from 12 to IH. h’ive factors were 
extracted from centroid analysis of the inter-test conclations, 
'Pile axes were then rotatetl orthogonally so as to preserve a 
positive manifold and, if possihlc, retain a general factor 
present in all the tests. It turns out, however, that no general 
factor is present. Three factors are immediate memory span 
(in digits forwards and digits backw.U'ds), vei-hal (in dis- 
sected sentences and vocahuhiry) and perceptual dexterity (in 
dissected sentences, matching, mazes) ; the two rrther factors 
are present m c<iual proportions in matching, arithmetic, and 
iTasoning. Koithcr of these two is (j as ordinarily operation- 
ally delined; one factor i.s the ability to lind or make a 
sigriilicunt pattern in a mass of irrelevant material, and the 
other factor is the ability of logical elimination, d’he sugges- 
tion is made that ij in this battery is complex and that orthodox 
tests of (j need ttr he constructed to preserve its functional 
unity. J.K.Kailin. 

Rolf, Merrill, “A Statistical Study of the Development of 

Intelligence Test Performance.” Journal of Psychology, 
XI (1941), 371-386. 

Using data available in the literature, correlations between 
test performance of children at a specific age and the gain in 
their performance one or more years later were estimated. 
The fact that the correlations showed no tendency to increase 
as the interval between test and retest increases indicates that 
the “Constancy of the I.Q." is due primarily to retention of 
earlier skills and knowledge rather than to correlations be- 
tween earlier scores and later increments. On the assumption 
that the I.Q. variability is constant, the same procedures were 
used to find correlations which would result if scores and later 
increments were uncorrelated. No comparison of these values 
and empirical findings is made, Lorraine Bouiliilct. 
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Schellhammer, Fred M. “The Intelligence Test in Teacher- 
Training Institutions.” School and Society, Till (1941), 
319. 

In a survey of 150 teacher-training institutions, 103 were 
found to use the intelligence test in student selection and 
evaluation of intclligeiice, 18 relying on it solely and 85 using 
it in conjunction with other techniques such as high school 
records and interview and faculty reports, no one combination 
finding universal favor. The majority of institutions con- 
sidered the high school record as important as the intelligence 
test, and were supplementing both measures with subjective 
techniques. Virginia Brown. 


Super, D. E. “A Comparison of the Diagnoses of a Graph- 
ologist with the Results of Psychological Tests.” Journal 
of Consulting Psychology, V (1941), 127-133. 

To check the claims of a woman “graphologist,” 24 stu- 
dents submitted samples of their handwriting and obtained 
the graphologist’s diagnoses. These were compared with the 
most appropriate of several test scores (Intelligence, Fryer 
& Sparling’s Occupational Intelligence Norms, Strong Voca- 
tional Interest, and Bernreuter Personality Inventory). Use 
of chi-square and other methods showed no more than chance 
relationship between occupations recommended and those in- 
dicated as suitable for intelligence scores obtained; occupa- 
tions rated as unsuitable by interest tests were recommended 
with more than chance frequency; personality traits were 
estimated by the graphologist with no more than chance 
agreement with test scores (on four traits), and worse than 
chance agreement (on two traits ) . F. A. Kingsbury. 


Thomson, Godfrey. “Critical Notice of ‘The Factors of the 
Mind’ by Cyril Burt.” British Journal of Educational 
/Wmio^y, XI (1941), 45-51. 

Thomson writes a brief review of Burt s most recent book 
( The Factors of the Mind, Univ. of London Press 1940, ^v 
+ 509). The major portion of the review considers Burts 
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section on the clistrilnition of temperamental types and the 
application of factor analysis to persons as well as to tests. 
Thomson docs not agree that the philosophical approach to 
factor Analysis is easier or more illuminating than the geomet- 
rical but he does express agreement with Burt’s conclusions as 
to the metaphysical status of mental factors. IF. A. Farvel. 

Traxlcr, Arthur E. and others. “Psychological 'Pests and 
Their Uses.” Reuiciu of Educational Research, XI 
(1941), 1-130. 

'Phis issue, consisting of eight papers not separately 
abstracted because of space limitations, is concerned with the 
construction, evaluation, and application of psychological 
tests. Individual articles are accompanied by extensive bibliog- 
raphies. The following is a list of authors and titles: 

1 “Brief Overview of the Period” (Arthur E. Traxler) 
II “Current Construction and Evaluation of Intelligence 
Tests” (Dewey B. Stuit) 

III “Applications of Intelligence Tests” (J. B. Stroud) 
[V “Measurement of Aptitudes in Specific Fields” (David 
iiegcl) 

V “Current Construction and Evaluation of Personality 
and Character 'Pests” (Arthur E. Traxler) 

VI “Projective Methods in the Study of Personality” 
(Percival M. Symonds) 

VII “Applications of Personality and Character Measure- 
ment” (John W. M. Rothney) 

VIII “Statistical Methods Related to Test Construction and 
Evaluation" (John C. Flanagan) D. A . Peterson. 

'Praxler, Arthur It, “Stability of Scores on the Primary Men- 
tal Abilities Tests.” School' and Society, LIU (1941), 
2SS. 

Test-rctest correlations after one year ranging, with one 
exception, from .578 to .917 were found for the scores of 104 
pupils in grades X-XII on Thurstone’s Primary Mental Abili- 
ties Tests. 'Phe guidance value of the perceptual, memory, 
and inductive tests may be limited, for their correlations fell 


426 



MEASUREMENT ABSTRACTS 


below .80. These results should be checked with a larger and 
more representative group as the sampling and number of 
cases were not adequate in the present study. Virginia Brown 

rim Use of Tests in the Illinois State Eniployment Service 
The Personnel and Training Section in collaboration with 
the Local Office Operations Section. Division of Place- 
ment and Unemployment Compensation, 205 West Wacker 
Drive, Chicago, Illinois. February, 1941. 11 pp. 

This pamphlet is intended to assist interviewers in the use 
of test results as a supplementary tool in “making more 
objective the evaluations which must be made during the inter- 
view.” The use of tests is related to other interviewer’s 
tools (i.e., Job Descriptions, The Dictionary of Occupational 
Titles, Registration and Placements Aids). The article de- 
scribes the types of tests, proficiency and aptitude tests, used 
in aiding the interviewer to evaluate work skills. Aptitude 
test batteries have been developed for three fields: selling, 
clerical work, and manual work. Three graphic illustrations 
of relation of scores on aptitude tests to job performance are 
given. D. A. Peterson 


Viteles, M. S. “A Psychologist Looks at Job Evaluation.” 
Personnel Journal, 'KNYL (1941), 165-176. 

The author recognizes the importance of job evaluation as 
a basic feature of the industrial relations program. In pro- 
moting the adjustment of workers, there is a need for a pro- 
cedure designed to establish an equitable basis of compensa- 
tion, to facilitate transfer and promotion, and to eliminate 
duplication of activities. The chief consideration of the paper 
is a critical examination of the various types of job evaluation 
programs in the light of psychological principles and experi- 
ence. Ways are indicated in which improvements might be 
effected through the application of the techniques and prin- 
ciples of applied psychology. The present trial and error 
approach could be converted into a * rational, logical, and 
scientific system of analysis.” W. A. V arvel. 
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Welch, Alfred C. “An Analj-tic System of Testinfj; Competi- 
tive Advertisinfr." Journal of Applied PsychaUujy, XXV 
(1941), 17d-ia9. 

This study is intended to correct the usual copy-test pro- 
cedure, which is defective in that it yields only a fTross evalua- 
tion, by combining into a uailicd system a number of different 
tests which will provitk; the advertiser witit clues to help him 
imprni-e his advertising An analytic system of testing com- 
petitive advertising was developed to provide a method of 
suggesting specific strong and weak aspects of an advertising 
campaign as well as to provide a gross evaluation of the effects 
of the campaign. TIic system is based upon four tests: A 
Brand Preference scale ( described previously) ; a Brand 
hTmlliarity scale (controlled association or aiilcd recall) in 
which the respondents were reijulred to name live brands in 
response to each of two stimulus-words, cigarettes and fountain 
pens; a 'riiemc Familiarity test (Link’s method of triple asso- 
ciates) in which the respomient must identify the sponsor of a 
particular advertising theme; and a 'riierne Credence test (a 
belief test that does not require the respondent to report 
directly whether he believes an advertising claim), d’ests ol 
reliability and validity for the various scales indicate that the 
Brand Familiarity, I'hernc Familiarity, anti 'Fhcine Credence 
Tests were useful supplements to the Brand Preference scale In 
analyzing the effects of advertising hut that none of the three 
tests could be depended upon as a valid measure if used alone. 
Examples of the use of the analytic system are given, /. h. 
Karlin. 

Wells, F, L. “Some Functions of Mental Measurements in 
the Young Superior Adult.” Journal of Consulting 
Psychology, V (1941), 1 OS- 110, 

A review of cases seen through the psychiatric division 
of the student health department in a large endowed univer- 
sity reveals about ten classes of adjustment problems. These 
are distinguished by different patterns of performance on the 
various s tanda^/d— e x - a m i n a t l a^-n — technlmies. Representative 
cases described. F. A. 
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